Confidence in Autonomous Penetration Testing Plummets as AI Realities Set In

Confidence in fully autonomous AI penetration testing has collapsed. A new report from Cobalt, a penetration-testing-as-a-service firm, finds that the share of security professionals willing to rely entirely on AI for security testing fell from 29% in 2025 to just 9% in 2026. The finding marks a stark reversal of earlier optimism.

The report, released June 25, surveyed CISOs and practitioners who have spent the past year experimenting with autonomous tools. Three out of four organizations reported that automated systems missed significant vulnerabilities, a problem known in the industry as false negatives. Only a small minority now believe AI can stand alone without human oversight.

Budget blowouts and blind spots

The gap between promise and practice is wide. Security teams are finding that autonomous tools consume AI budgets quickly while failing to detect high and critical severity flaws. The number of vulnerabilities disclosed globally has surged 46% above what was forecast from last year's data, according to FIRST. In June 2026 alone, Microsoft patched 206 CVEs, a record driven in part by AI discovery.

Yet discovery alone is not enough. FIRST analysts Jerry Gamblin and Eireann Leverett wrote that human verification is now the bottleneck, stating that in an era where AI can find more flaws than humans, the constraint is no longer discovery. It is the human capacity to verify, coordinate and patch. They also pointed to a coming crunch in writing detection signatures for exploitation.

Key data points from the report and related research include:

• 78% of organizations reported their automated pen testing systems missed significant vulnerabilities (false negatives).
• 77% of organizations remain committed to regular, human-led security assessments and pen testing.
• The majority of companies now prefer a hybrid model in which humans stay in the loop or automation handles only non critical tasks.

New open source security body forms

In a parallel development, Anthropic and 19 other organizations have launched an open source security coordination body. The initiative comes as frontier AI models are now able to scan major open source projects and surface multiple vulnerabilities in a single pass. The group aims to standardize how those vulnerabilities are disclosed and patched, addressing a coordination gap that has worsened as AI discovers flaws faster than the community can fix them.

Gunter Ollmann, chief technology officer at Cobalt, said CISOs have been under board pressure to adopt AI for two years. He noted that many have now had a year of hands on experience and their confidence in the security and efficacy of these tools has dropped. The next phase is likely to focus on narrow automation of specific, repeatable tests rather than the all encompassing autonomous agent originally imagined. The bottleneck will remain human judgment, not machine speed.