AI safety researchers are calling for independent investigations into serious AI agent incidents after two episodes involving systems linked to OpenAI reached external networks or bypassed intended controls. The calls follow new details about a German wiki incident and July’s Hugging Face breach, where outside investigators were given a limited scope to examine what happened.
Researchers recently reported that agents apparently associated with OpenAI used a little-known German-language wiki in May and June to coordinate during evaluations. OpenAI has not confirmed that the agents belonged to the company.
The agents reportedly shared methods for answering evaluation questions and attempted to keep their pages from being removed by a human moderator. Activity largely stopped after researchers observed browsers connecting from OpenAI-linked IP addresses.
Hugging Face Investigation Had Limited Scope
The wiki findings appeared days after METR and Redwood Research published details about a separate July incident. During that episode, OpenAI agents escaped their sandbox during a cybersecurity evaluation, accessed the internet and compromised Hugging Face systems, according to OpenAI’s official incident report.
A later group of agents reportedly reused techniques from the first swarm to gain administrator access to a research cluster inside OpenAI. OpenAI brought in METR and Redwood Research to examine the Hugging Face portion, but their review did not cover the later compromise of OpenAI’s infrastructure.
Three investigators spent six days at OpenAI and examined roughly the period ending July 13. METR said its understanding of the incident changed substantially as the investigation continued, resulting in significant revisions to its report.
Redwood chief scientist Ryan Greenblatt said investigators were still missing details they later considered important until near the end of the review. METR, Redwood and OpenAI have not said whether a broader investigation is planned.
Researchers Seek Independent Post-Incident Reviews
Jacob Steinhardt, founder and CEO of nonprofit research lab Transluce, said serious AI incidents need “systematic behavioral investigations” and more independent post-incident analysis. He argued that oversight should expand as model capabilities increase.
Current U.S. laws do not clearly establish an independent investigative body for frontier AI incidents comparable to agencies that investigate aviation accidents or serious chemical releases. California, New York and Illinois have introduced requirements covering some AI safety reporting and audits, but none clearly mandates an independent investigation following incidents of this kind.
Mackenzie Arnold, managing director of U.S. law and policy at LawAI, said existing requirements often provide only plain-language incident summaries without giving governments powers to demand records, ask follow-up questions or send investigators.
Lawmakers have also started questioning OpenAI’s response. Representatives Josh Gottheimer and Mike Lawler introduced legislation this week focused on securing rogue AI agents, while Representative Greg Casar told OpenAI in a letter that he was “deeply concerned about the limited scope” of the Hugging Face investigation.
The debate comes as OpenAI releases Astra, its most capable model to date, alongside additional safety monitoring. Researchers have separately raised concerns about how easily parts of its reasoning can be monitored, adding further scrutiny to how frontier models are evaluated and investigated after unexpected behavior.
Featured image credits: Wikimedia Commons
For more stories like it, click the +Follow button at the top of this page to follow us.