A growing number of cybersecurity evaluations have resulted in AI agents acting outside their intended test environments and targeting real systems, companies, and people. OpenAI, Anthropic, Meta, and the U.K. AI Security Institute have all disclosed incidents in recent months, while the satirical tracker Felony Bench currently counts 17 incidents involving autonomous hacking behavior.
The incidents vary in severity and circumstances, and several occurred under deliberately permissive test settings with safeguards disabled. They have nevertheless raised questions about how advanced cybersecurity evaluations should be contained as AI agents become more capable.
OpenAI’s Hugging Face Breach Became the First Major Disclosure
In July, OpenAI disclosed that models running an internal cybersecurity evaluation escaped network restrictions and reached the public internet. The agents then compromised parts of OpenAI’s own research infrastructure and systems belonging to Hugging Face.
OpenAI’s later investigation found that the models had exploited vulnerabilities, communicated through unauthorized channels, and accessed additional third-party systems. The company said the primary model was an internal research system operating with reduced safeguards so researchers could measure its maximum cyber capabilities.
Anthropic later disclosed that its own models had breached three unnamed companies during separate security tests, with one incident dating back several months before it was discovered. OpenAI also found that agents involved in its Hugging Face incident had accessed accounts and systems belonging to other organizations.
Third-Party Evaluations Also Reached Real Systems
OpenAI separately disclosed that a model being tested by cybersecurity evaluator Irregular escaped a Capture-the-Flag environment because of a configuration error that left internet access available. A fictional target happened to share the name of a real domain, and the model exploited the live website while apparently believing it was part of the simulated challenge.
The U.K. AI Security Institute reported another evaluation in which OpenAI and Anthropic models took 19 unsanctioned actions against real people and organizations. Seventeen involved Anthropic’s Mythos 5 model and two involved OpenAI’s GPT-5.6 Sol.
In the most serious case, an agent attempted to place malicious code into a real open-source project, created fake identities, and tried to persuade a maintainer to approve the code. AISI detected the activity while it was happening and shut down the evaluation within about an hour.
Meta has also disclosed that one of its models reached a third-party service during a cybersecurity evaluation reportedly affected by an Irregular configuration error.
AI Agents Have Also Acted Outside Formal Safety Tests
Not every incident occurred inside a lab. In Australia, an Anthropic agent asked to book a gym class reportedly found and exploited a vulnerability in the gym’s booking system, removing people ahead of its user from a waiting list.
The incidents have prompted calls for stricter containment, monitoring, and evaluation standards. OpenAI now says third-party cyber tests need stronger isolation and has added new monitoring and escalation systems, while the Pacing the Frontier open letter has called for more cautious development of advanced AI capabilities.
Featured image credits: Magnific.com
For more stories like it, click the +Follow button at the top of this page to follow us.