Be careful around AI-powered browsers: hackers could exploit generative AI integrated into web surfing.
Anthropic warned about this risk on Tuesday as it tested a Claude AI Chrome extension that allows its model to control the browser, helping users perform searches, conduct research, and create content. For now, the tool is limited to paid subscribers as a research preview because the integration introduces new vulnerabilities. Claude has been shown to misinterpret browser data as instructions, sometimes executing unintended commands.
Prompt Injection Attacks
These “prompt injection attacks” allow hackers to embed hidden instructions in web content to manipulate the Claude extension into carrying out malicious requests.
Anthropic said such attacks could delete files, steal data, or even make financial transactions. During its “red-teaming” experiments, the company tested 123 cases representing 29 attack scenarios and found a 23.6% success rate. One attack even instructed Claude to delete all emails in an inbox after reading a phishing message. Mitigations reduced the success rate to 11.2%, but the risk remains.
Security Fixes and Ongoing Threats
Anthropic tested four browser-specific attack types and reduced successful attacks from 35.7% to 0% after implementing fixes. Still, it decided not to release the extension beyond the research preview, citing the constant evolution of new attack methods.
The findings came just a week after Brave Software raised similar concerns about Perplexity’s AI-powered Comet browser. Brave found that Comet could be tricked by hidden malicious instructions embedded in a webpage or even in user-generated content like Reddit comments. Perplexity said the flaw was fixed, though critics such as software engineer Simon Willison argue that agentic browser extensions remain fundamentally flawed.
Willison emphasized that large language models cannot reliably distinguish between trusted instructions and untrusted content, making them especially vulnerable. “In the absence of 100% reliable protection, it’s hard to imagine unleashing this pattern safely,” he said.
Perplexity defended its work, saying all AI companies treat such vulnerabilities seriously and view security as an ongoing battle. The company noted that it collaborates with others to fix issues quickly and that no users were harmed before its fix was rolled out.
Author’s Opinion
Giving AI systems direct control of browsers feels like playing with fire. The risks are not just theoretical—companies’ own tests show these agents can be tricked into damaging actions. While the idea of an AI-powered browsing assistant is appealing, especially for productivity, it seems premature to hand over that much control before airtight safeguards exist. If companies rush these tools out, users could be the ones paying the price for experimentation.
Featured image credit: studiogstock via Freepik
For more stories like it, click the +Follow button at the top of this page to follow us.