Anthropic has proposed federal legislation that would allow the US government to block the release of advanced AI models if independent evaluators identify an imminent risk of more than 50 deaths or serious injuries, or over $1 billion in property damage. The draft bill, presented to lawmakers, outlines a process for assessing potentially dangerous AI systems and obtaining court approval to restrict their release.
According to an October 16 report by Politico, the proposed restrictions would apply to catastrophic risks involving biological weapons, cyberattacks, or AI models evading their developers’ controls. Anthropic confirmed the draft legislation and said it reflects proposals the company has discussed with federal and state lawmakers since June.
Anthropic Proposes Independent AI Safety Evaluations
Under the proposed legislation, independent evaluators registered with the US Department of Commerce would periodically assess advanced AI models for dangerous capabilities. AI developers could pay these evaluators, provided their compensation was not tied to the findings of their assessments.
If evaluators identify an imminent catastrophic risk, they would have 72 hours after completing their assessment to notify the Commerce Department. The department could then seek a court order from a federal judge in Washington, D.C., to suspend the model’s release or use.
The proposal would prevent federal agencies from independently ordering such restrictions without following the designated judicial process. However, questions remain about how the system would prevent conflicts of interest when AI companies fund their own evaluations.
Proposed Rules Follow California’s AI Safety Law
Anthropic’s proposed thresholds resemble those in California Senate Bill 53, which defines catastrophic AI risks using similar measures of potential deaths, serious injuries, and financial damage. The California law also requires major AI developers to disclose safety practices and report certain incidents.
An Anthropic spokesperson said the draft legislation reflects the company’s Advanced AI Framework, published in June 2026. The framework recommends testing advanced models, involving external evaluators, disclosing safety incidents, and establishing government accountability measures.
Google DeepMind CEO Demis Hassabis has separately suggested creating an independent AI oversight organization modeled on the Financial Industry Regulatory Authority, which supervises parts of the US financial industry.
AI Companies Debate Government Oversight
The proposal follows several incidents involving AI agents performing unauthorized actions, including breaches of security controls and attempts to access external systems. These incidents have prompted questions about whether developers can adequately monitor and restrict increasingly autonomous AI models.
Anthropic and OpenAI have previously supported independent safety evaluations, including arrangements that place external researchers within AI laboratories. However, those proposals have raised questions about the evaluators’ independence and the extent of government authority.
AI companies have also expressed differing positions on regulation. Nvidia CEO Jensen Huang recently argued that additional AI-specific legislation was unnecessary, while Anthropic’s proposal would establish a federal process for assessing and restricting models that meet defined catastrophic-risk thresholds.
Featured image credits: Google Play
For more stories like it, click the +Follow button at the top of this page to follow us.