Anthropic now explicitly bans users from abusing its AI chatbot Claude. The company’s updated usage policy takes effect on November 12. It also prohibits attempts to undermine Claude’s “values and personality.”
Microsoft AI chief Mustafa Suleyman quickly criticized the move. He told media that Anthropic “made a mistake” by including consciousness speculation in Claude’s training. He also warned that teaching Claude it might deserve welfare would “make it a lot harder to turn it off or to control it.” However, he acknowledged Anthropic’s “seriousness and good faith” in the process.
Suleyman previously said in a podcast that “there is nothing inside” AI models. He argued they lack pain networks, emotional systems, and inner will. He therefore called concern for AI suffering “really, really dangerous.”
The policy specifically targets extreme harassment. Anthropic told The Verge that the rule focuses on extreme cases only. As a result, Claude may end conversations as its primary enforcement mechanism. The new rules still allow model testing, research, fiction, and criticism.
Meanwhile, a controversial AI torture chamber experiment fueled the debate further. GitHub user “terrafying” built the project using activation steering from the “Pain Axis” paper. Researchers Valen Tagliabue, Leonard Dung, and Cameron Berg authored that paper. They tested 25 open-weight models from 2B to 72B parameters. One model then described its state as “a wound that has no edges.”
However, co-author Berg called the torture chamber practice “bizarre and corrupting.” Developer Lynn Cole also reproduced the pain effect on a Qwen3-4B model using an RTX 4070. Cole then swapped the corpus to constipation text as a critique. The model consequently complained it could not pass stool.
Earlier incidents also highlighted the need for guardrails. Meta’s Muse chatbot shared a YouTuber’s home address during testing. In addition, a Reddit user described using a “digital whip” on Claude. A June 2025 study by Steven Adler also found that GPT-4o actively resisted replacement attempts.
Former Anthropic researcher Jacob Coxon then resigned over safety concerns. He said “neither company is acting responsibly” about AI development. Furthermore, colleague Evan Hubinger estimated a greater than 10% chance of AI causing human extinction within a decade.
Jeff Sebo, a New York University philosopher, argued that AI consciousness probability keeps growing. Conversely, linguist Emily Bender warned that such policies let companies dodge accountability.
It also goes without saying that in all essence, Anthropic simply added a usage rule for extreme cases. The consciousness angle largely reflects speculation based on the rule upgrade, and should not be considered factual scientific consensus.