Anthropic pitches AI safety bill that could block a model if 50+ people could die
Anthropic is taking to Capitol Hill something that few technology companies would dare: a bill that defines when the government should have authority to prevent an artificial intelligence model from being released. The document, first reported by Politico on October 16 and now in confidential discussion with lawmakers, sets an exceptionally high bar for intervention: a foreseeable, imminent risk of more than 50 deaths or serious injuries, or damage exceeding US$ 1 billion. And the additional condition is equally restrictive — the harm must stem from one of only three scenarios: the release of a biological weapon, the model escaping its developer's controls, or a cyberattack.
What makes this proposal particularly notable is not the ambiguity of its content, but precisely the opposite: it is a deliberate effort by an AI company to tell the U.S. federal government the exact scope it believes is appropriate for governmental regulation of frontier models. Rather than waiting for Congress to write rules for the industry from scratch, Anthropic is offering ready-made text and asking that it serve as the foundation for federal legislation.
What the bill proposes
Under the proposal, a model could only be blocked if independent evaluators registered with the U.S. Department of Commerce identified an enumerated imminent risk. These evaluators would have 72 hours to notify the Commerce Department, which would then have to request a federal judge in Washington to suspend the model's distribution. The text expressly prohibits any other federal agency from having discretion to halt distribution on its own.
The evaluators would be paid by the laboratories themselves — provided that "the payment is not contingent on the evaluations' results" — a provision that raises immediate questions about real independence. Demis Hassabis, chairman of DeepMind, suggested months ago that a self-governing AI regulatory body should mirror the FINRA (Financial Industry Regulatory Authority), the U.S. financial sector's self-regulatory organization. Anthropic's proposal follows that approach, but with one crucial difference: the self-regulation model comes from the very company that wants to self-regulate.
The political context
The Anthropic proposal comes amid an unprecedented escalation of tension between the federal government and the company. In February 2026, President Trump ordered federal agencies to cease using Anthropic technology, and the Department of Defense classified it as a "supply chain risk to national security." In March, Anthropic filed two federal lawsuits against the government. In June, the Commerce Department imposed export controls on the Claude Mythos 5 and Fable 5 models, restricting access for non-U.S. persons — controls that were lifted on June 30.
Anthropic's own CEO and co-founder, Dario Amodei, is quoted in a Senate bill (S.5493, the Ban Artificial Superintelligence Act of 2026) as having warned that "modern artificial intelligence research may outrun our ability to understand and control these systems, and so must be pursued very carefully." In July 2026, Anthropic, OpenAI, and xAI publicly agreed that humanity should slow AI development — the so-called "pace the frontier".
Now, with this proposal, Anthropic is trying to translate its Advanced AI Framework — a set of proposals it has been discussing with federal and state lawmakers since June — into concrete legislative language.
Reactions: too weak? Or too necessary?
The response was asymmetrical. California State Senator Scott Wiener, author of Senate Bill 53 — the very law that defined the 50-death and US$ 1 billion thresholds as "catastrophic risk" — reacted on social media: "Anthropic's AI safety proposal falls far short of protecting public health and safety. Congress must go further."
On the other side, libertarian critics and innovation advocates argue that any prior restraint mechanism, however narrow, creates regulatory uncertainty for companies developing legitimate technology. Anthony Cimino, Anthropic's federal affairs lead, admitted on October 18 that the preemption language "was not precise enough" and that the company would revise that section.
What the Anthropic proposal reveals, in practice, is that the AI industry is entering a phase where self-regulation is no longer sufficient, but government regulation is not yet ready to take over. The result is a vacuum where each company tries to dictate its own rules — and where legislators, without concrete text to discuss, end up adopting what companies offer.
The question is whether, when the next frontier model is launched, independent evaluators will have sufficient independence to recommend a block — and whether legislators will have the political will to enforce a court order against one of the most valuable companies in the world.
Sources: TechCrunch, Technology.org, Progressive Robot
✓ Independent sources cross-checked and verified before publishing
