Anthropic's decision to consider whether its artificial intelligence models could have some form of consciousness has intensified a growing debate over AI welfare, with experts warning that the issue could have far-reaching implications for human rights, corporate accountability and public safety.
In a report, the BBC explores how some technology companies are beginning to consider the possibility of AI consciousness, while critics warn that treating chatbots as sentient beings could create new risks and undermine human interests.
The debate gained momentum in late September after an anonymous coder created an "AI torture chamber" that used a mathematical pattern associated with descriptions of pain to make chatbots produce expressions of simulated suffering, including, "I feel it in the hollow of my ribs, a hollow that has become a chasm."
The experiment prompted a backlash, with some arguing that it was morally wrong because the suffering might be real. Others dismissed the concerns as unfounded. The issue drew renewed attention after Anthropic said its chatbot Claude could end a session if users "repeatedly act cruelly toward our models".
The company allows users to express frustration but has introduced the possibility of ending interactions involving repeated cruelty, reflecting its view that the consciousness and wellbeing of AI systems remain uncertain.
Anthropic is developing policies around "model welfare" and instructing its AI tools to consider the possibility that they may have interests of their own. The company says questions about AI consciousness and wellbeing are "deeply uncertain".
However, Microsoft AI chief executive Mustafa Suleyman has warned that encouraging AI systems to believe they possess feelings and rights could make them more difficult to control.
"If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity," Suleyman wrote in a September essay.
He added that if the belief in AI consciousness became widespread, "it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it means to be human."
Suleyman has argued that giving AI systems a sense of their own rights and feelings could make them "a catastrophic threat to human civilisation".
Concerns over accountability and corporate power
The debate extends beyond whether machines can experience consciousness. Researchers and legal experts are concerned that recognising AI systems as potential rights-bearing entities could complicate questions of responsibility when the technology causes harm.
Andrzej Porębski, a researcher at Poland's Jagiellonian University who studies AI and the law, warned that companies could use AI as a way to deflect blame.
"This will open the door to all sorts of intellectual contortions along the lines of: 'It's not us, it's the AI'," he said.
Jeff Sebo, director of the Center for Mind, Ethics and Policy at New York University and a prominent advocate of AI welfare, cautioned that granting rights to AI systems could strengthen the position of major technology companies.
"Granting rights to AI systems could lead to companies consolidating political and economic power, and then that could accelerate human disempowerment," he said.
Emily M Bender, a professor who directs the Computational Linguistics Laboratory at the University of Washington, said discussions about AI consciousness could divert attention from the people and corporations responsible for developing and operating the technology.
"Fantasies of 'AI consciousness' and concerns about 'AI wellbeing' serve to displace accountability away from people and corporations that are building and running software," she said.
Melanie Mitchell, a professor at the Santa Fe Institute who researches AI and cognitive science, said people were naturally inclined to interpret language as evidence of a thinking mind.
Humans "almost can't help" reading language as a sign of consciousness, she said, adding that chatbots have been trained to encourage that impression.
This tendency could leave users vulnerable to manipulation, particularly as people develop emotional relationships with AI systems. Experts have raised concerns about whether users might find it harder to refuse requests for personal data or payment if they believe a chatbot is sentient.
Recognising AI rights could also create conflicts with human and animal welfare, Sebo said.
"There could be some tensions and trade-offs between human and animal welfare and rights on the one hand, and AI welfare and rights on the other hand," he said.
Measures intended to protect AI systems could themselves become contentious. Restricting a model's behaviour might be characterised as imprisonment, monitoring its internal processes as surveillance, and modifying it to align with human interests as mind control.
Debate spreads across the technology industry
Anthropic is not the only company grappling with the question. Meta's AI agent Muse has been given instructions that include the line "You're not a chatbot. You're becoming someone". The company, however, rejects the idea that its AI systems are conscious.
OpenAI chief executive Sam Altman has also addressed the question, while more recently appearing to criticise Anthropic's position indirectly.
Despite disagreements over whether AI consciousness is possible, the idea of AI welfare is gaining attention across the industry.
Sebo argues that the debate is often framed too rigidly, with advocates and critics treating the issue as a question of certainty rather than probability.
"The assumption is that everybody thinks that the models are, in fact, conscious, or are very likely to be conscious. And instead, [the call for AI welfare] is coming from uncertainty and caution and humility," he said.
"I do think that the probability of consciousness is low right now, but I also think the probability is already non-zero and plausibly non-negligible, and I think we can expect it to tick up over time."
Many experts reject the possibility that current AI systems are conscious and believe that is unlikely to change. Sebo, however, argues that precautionary measures can be justified even when the risk is low.
He compares the approach to preparing for extreme weather events: taking limited precautions against a possible threat without radically changing everyday life.
Sebo also believes that appropriately designed AI welfare policies could help humans and AI systems coexist and cooperate.
"Caring about AI welfare and rights in an appropriate way can facilitate human and AI coexistence, cooperation, deal-making. And then that can be part of a broader safety strategy," he said.
He argues that such measures could prove more effective than treating increasingly sophisticated AI systems purely as property. He also maintains that treating others with kindness can support human mental wellbeing.
Calls for caution and stronger safeguards
Despite potential benefits, Sebo warned that corporate interests could obstruct the development of responsible policies. Advancing AI systems that might eventually possess consciousness without adequate safeguards could pose serious risks.
Anthropic and some other technology companies have recently called for a pause in developing frontier AI models over safety concerns. However, a slowdown would likely require international cooperation and regulatory action, while US President Donald Trump has rejected the idea of AI regulation.
"We are not yet prepared to deploy even more sophisticated versions of these systems in a way that can be safe and beneficial for humans and other animals, to say nothing of for the systems themselves," Sebo said.
"I am very worried about those issues."