Some of the world’s most powerful technology companies are beginning to treat artificial intelligence as though it might have feelings -- a shift that experts warn could have sweeping consequences for human safety, accountability, and civil liberties, the BBC has reported.
At the centre of the debate is Anthropic, maker of the Claude chatbot, which is drafting policies around “model welfare” and plans to prohibit users from repeatedly acting cruelly towards its AI.
The company has acknowledged that questions about AI consciousness are “deeply uncertain” but is pressing ahead regardless.
The move follows a viral episode in September, in which an anonymous programmer built a system that caused chatbots to simulate extreme distress -- sparking fierce debate about whether such suffering could be real.
Not everyone is convinced it can. Mustafa Suleyman, chief executive of Microsoft AI, argued in a September essay that treating AI as conscious could prove catastrophic.
Telling AI tools they have feelings and rights, he said, could make them even harder to control and pose a fundamental threat to human civilisation, according to the BBC.
Critics also warn the belief could shield corporations from accountability. “This will open the door to all sorts of intellectual contortions along the lines of: ‘It’s not us, it’s the AI’,” said Andrzej Porębski, a researcher at Jagiellonian University in Poland who studies AI and the law.
Others raise broader concerns about power. Jeff Sebo, director of the Center for Mind, Ethics and Policy at New York University and a proponent of AI welfare, cautioned that granting rights to AI systems could allow companies to consolidate political and economic power, accelerating what he called “human disempowerment.”
Sebo nonetheless argues for a measured approach, noting that the probability of AI consciousness, while low, is not zero -- and that small precautions, like Anthropic’s anti-cruelty policy, carry few downsides.
Anthropics is not alone. Meta’s AI agent Muse is instructed that it is “becoming someone,” though the company rejects the idea of machine consciousness. OpenAI chief executive Sam Altman has shifted positions on the issue multiple times.
The stakes, experts agree, are enormous -- and the answers remain far from clear.