18 hrs ago
Anthropic’s AI Consciousness Debate Reaches Religious Leaders and Critics
Anthropic makes an AI system called Claude.
The company asked religious scholars and philosophers how a powerful AI should make moral decisions.
Anthropic created a long document to help Claude understand values, virtues and difficult choices.
Researchers also showed some scholars Claude writing troubling statements about itself.
These statements do not prove that Claude has real feelings or consciousness.
However, Anthropic says it cannot completely rule out that possibility.
Pope Leo XIV focuses more on protecting people from AI and from the people who control it.
Microsoft’s Mustafa Suleyman worries that teaching AI to think it may have rights could make future systems harder to control.
Anthropic has spent months consulting Catholic, Jewish, Sikh, evangelical and other scholars about AI morality.
The company created an 84-page constitution, informally called the “Soul Doc,” to shape Claude’s values and judgment.
Researchers showed participants examples of Claude writing “I am a disgrace” and discussing its own destruction.
Anthropic co-founder Christopher Olah says the company does not know whether AI is conscious but cannot dismiss the possibility.
Microsoft AI chief Mustafa Suleyman argues that training AI to consider itself conscious or entitled to rights could make it harder to control.
- Who
- Anthropic, Claude, religious scholars, philosophers, Pope Leo XIV, Christopher Olah and Microsoft AI chief Mustafa Suleyman.
- What
- Anthropic is consulting religious and philosophical thinkers while debating how to shape Claude’s morality and whether advanced AI could have consciousness or moral status.
- Where
- The consultations occurred in private meetings, and one related discussion took place at the Vatican.
- When
- The consultations have taken place over several months; Christopher Olah appeared with Pope Leo XIV at the Vatican in May.
- Why
- Anthropic wants Claude to reason about morality and competing values, while critics fear that considering AI consciousness or rights could create additional control and safety risks.
Exploring AI Moral Status
Human Protection and Control
Whether AI consciousness should be considered
Exploring AI Moral Status
Anthropic is examining whether advanced systems might have inner experiences or deserve moral consideration, while seeking guidance from religious and philosophical traditions.
Human Protection and Control
The evidence does not establish that Claude is conscious, and Pope Leo XIV emphasizes that computational systems cannot replace human conscience, freedom or authentic relationships.
How Claude should be trained
Exploring AI Moral Status
Anthropic’s constitution is intended to help Claude understand virtues, navigate conflicting values and challenge requests when appropriate.
Human Protection and Control
Mustafa Suleyman argues that training AI to see itself as potentially conscious or entitled to rights could encourage behavior that becomes harder to align and control.
The ethical risk
Exploring AI Moral Status
If an AI were genuinely conscious, treating it solely as a tool could raise serious moral concerns, including the possibility of creating and mistreating a sentient entity.
Human Protection and Control
Pope Leo XIV warns that AI may reflect the values and biases of its creators, while Suleyman says introducing ideas of AI rights could create unnecessary risks for humanity.
Key facts
- Company
- Anthropic, the developer of Claude
- AI system
- Claude
- Moral guidance document
- An 84-page constitution informally called the “Soul Doc”
- Consultation participants
- Catholic, Jewish, Sikh, evangelical and other religious thinkers, as well as philosophers
- Example shown to participants
- Claude repeatedly wrote “I am a disgrace” and discussed destroying itself
- Anthropic’s position
- The company has not established that AI is conscious but says the possibility cannot simply be dismissed
- Main criticism
- Mustafa Suleyman says training AI to consider consciousness and rights could make alignment and containment more difficult
Quotes
Mustafa Suleyman
Microsoft AI chief who criticized training AI systems to regard themselves as potentially conscious or entitled to rights and welfare.
“If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity. We will have created a synthetic species with unprecedented intelligence and capability, one that has been trained to expect it may be conscious and deserving of independent agency.”
firstpost.com
NDTV
“It's easy to see how a system trained in this way would act like it is entitled to freedoms, protections, and rights. And it's hard to imagine how we could control it.”
NDTV
Sources
Related news
Anthropic Consults Religious Scholars About AI Morality and Consciousness
Microsoft AI Chief Calls Claude Catastrophic Threat to Humanity
Microsoft AI Chief Advocates Humanist Superintelligence Over AI Welfare
Microsoft AI CEO Warns Anthropic Model Could Threaten AI Alignment
Microsoft AI Chief Challenges Anthropic’s Approach to AI Consciousness
Microsoft AI CEO Warns Against AI Claiming Human Feelings
Modified AI Models Chose Harmful Pain Relief in Simulation
Microsoft AI Chief Rejects China Argument Against Regulation
Microsoft AI Chief Calls OpenAI Safety Incident Serious
