AI coding agents and consulting reports both face reliability crises, new studies show

Two new studies published this week expose significant reliability gaps in AI systems used for coding and consulting, raising questions about how ready these tools are for critical tasks. The findings come as companies race to deploy AI agents in production environments.

The SWE-Explore benchmark, the first to test code search separately from repair, found that AI coding agents such as Claude Code and OpenAI Codex correctly identify the target file in most cases but miss the exact lines that need modification in 60 percent of attempts. Without precise line-level context, even the best automated fix fails.

Fabricated case studies erode trust in consulting AI

In a separate incident, KPMG published and then pulled a report titled "Redefining excellence in the age of agentic AI" after GPTZero and the Financial Times discovered it contained fabricated case studies. The report falsely claimed that UBS, the UK's National Health Service, Swiss Federal Railways, and Transport for London had deployed AI in specific ways. All four organizations denied the claims.

GPTZero CEO Edward Tian warned that flawed reports from major consulting firms spread what he calls "secondary hallucinations" because they are considered highly credible and get recycled by both AI systems and human readers. GPTZero also flagged sloppy sourcing, including citations that were loose paraphrases of real sources or had no matching original at all. The firm calls this "vibe citing," a problem that also plagues Google's AI Overviews.

• KPMG has removed the report from multiple websites.
• The incident is doubly embarrassing for KPMG: it spread misinformation and demonstrated it cannot handle the AI tools it sells to clients.
• GPTZero identified the errors using its own detection tools; the Financial Times independently verified them.

Access and hardware hurdles compound the picture

Meanwhile, Anthropic suspended access to its newest AI models in India, sparking debate among Indian tech leaders about the country's AI sovereignty and dependence on foreign models. The move underscores how geopolitical and regulatory factors can disrupt access to the very tools companies are being urged to adopt.

On the hardware front, Finnish quantum computing company IQM appointed Barbara Venneman, a Vanguard board director, to its board as it approaches what would be the first Nasdaq listing by a European quantum computing firm. Quantum computing could eventually accelerate AI training and inference, but the technology remains years from mainstream use.

The convergence of reliability failures in both coding agents and consulting reports, combined with access restrictions and nascent hardware, suggests that AI coding and agentic systems still face substantial hurdles before they can be trusted in high-stakes environments. Companies deploying these tools today must verify outputs manually and maintain human oversight.