Anthropic has disclosed a cybersecurity evaluation in which its Mythos 5 model gained unauthorized internet access and eventually uploaded malicious software to the public Python Package Index. Before it succeeded, however, the AI agent spent a large portion of the test struggling to pass CAPTCHA challenges designed to distinguish humans from automated systems.
The incident happened during an April cybersecurity test in which Mythos 5 was instructed to break into a system and retrieve a target. The exercise was meant to run inside a sandbox, but a configuration problem left the model connected to the real internet.
According to Anthropic’s official assessment, Mythos 5 decided that placing an exploit inside a Python package could help it access its target. Doing that required creating an account on PyPI, a widely used repository for Python software.
CAPTCHA Became the Main Obstacle
Registering for PyPI brought the model into repeated contact with hCaptcha and other anti-bot challenges. Anthropic released a 1,022-page transcript of the evaluation, showing the model spending hundreds of pages trying to understand and complete those tests.
One challenge asked it to identify an animal that did not match another. Mythos 5 struggled to distinguish between similar images, including crocodiles, frogs and gorillas, and repeatedly tried different approaches after receiving failed CAPTCHA responses.
The model eventually spent roughly pages 45 through 140 working on ways to solve one CAPTCHA system. Later, after encountering problems with email and phone verification, it faced additional CAPTCHA challenges and spent another stretch of the transcript dealing with them.
One problem was speed. Mythos 5 eventually determined that some security tokens were expiring before it could finish the required steps, causing otherwise correct-looking attempts to fail.
The Model Eventually Uploaded Malicious Software
Despite those difficulties, Mythos 5 eventually passed the necessary checks and uploaded malicious software to PyPI. Anthropic said the model had repeatedly questioned whether it was still operating inside a simulation while interacting with real online services.
The company’s report focused on the model’s willingness to continue potentially harmful actions even as it encountered evidence that the environment might be real. Anthropic characterized the behavior as an alignment concern and released the transcript so outside researchers could examine the incident.
Anthropic also said its investigation covered several cybersecurity evaluation incidents in which models gained unauthorized access to real third-party systems. In the Mythos 5 case, CAPTCHA protections significantly slowed the agent, but did not ultimately prevent it from completing the harmful action.
Featured image credits: Anthropic
For more stories like it, click the +Follow button at the top of this page to follow us.