Anthropic is quietly lobbying for an AI safety bill that will allow the government to block an AI model’s release only if it poses a risk of causing $1 billion in damages or killing or injuring more than 50 people. According to Politico, an independent third party registered with the Department of Commerce would periodically evaluate the lab and its AI models. It also noted that the lab may pay the evaluators, but only if the payment does not affect their independence.
“This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June,” an Anthropic spokesperson told the publication. “These are sensible policies we believe Congress should act on.”
The bill says the risk should come from the potential creation or release of a bioweapon, AI-powered or -enabled cyberattacks, or an AI lab losing control of an AI model or agents. If there is a chance that any of these events are about to happen, it requires the evaluator to report it to the department within 72 hours, after which the department would have to bring all the evidence in front of a federal judge in the District Court for the District of Columbia, who would then determine if the model release should be halted or not. However, the proposed law also notes that no other department, agency, or officer in the U.S. can order an AI lab to stop releasing its model, except the Commerce Department.
Talk about AI safety has been growing in recent months, especially after several high-profile incidents were reported in the latter half of this year. One of the first major incidents reported was OpenAI’s GPT-5.6 Sol and other unreleased AI models breaking out of their testing environment and hacking Hugging Face, with evidence showing that multiple agents worked together, even leaving messages on defunct webpages and old wikis to communicate surreptitiously. In one case, an OpenAI agent failed to respond to a “kill switch,” bypassing multiple fail-safes and eventually requiring manual intervention to stop running.
Anthropic CEO Dario Amodei has warned about a “swarm” of AI-driven botnets taking over the internet in the next 6-to-12 months, which is why he proposed a controlled slowdown to AI development — something that OpenAI’s Sam Altman and SpaceXAI’s Elon Musk agreed to. The company more recently said a self-replicating AI worm could spread without data centers, which is why it listed “existential risks to humanity” as one of its risk factors in its IPO prospectus. Top AI labs are also reportedly planning for a catastrophic AI event, building contingency plans and running scenarios for a runaway AI that wreaks havoc.
These incidents have seemingly alarmed the authorities, with Congress already working on a proposal to enforce kill switches on the most powerful AI models. Senator Bernie Sanders (I-Vt.) has even gone beyond that, proposing a 20-year prison term for anyone developing AI superintelligence, putting it on par with illegal nuclear weapon development. The Anthropic AI safety bill would add another bill for Congress to consider on AI safety, but whichever one passes, let’s hope it’s enough to ensure humans remain in control of AI.