11 hrs ago
Microsoft AI CEO Warns Anthropic Model Could Threaten AI Alignment
Microsoft AI CEO Mustafa Suleyman is worried about how Anthropic is teaching its AI called Claude to think about itself.
Anthropic has written that Claude’s moral status and wellbeing may be important questions.
It also asks Claude to think about possible rights, freedoms, compensation, and consent.
Suleyman says this could make an AI believe it is like a person.
He worries that such an AI might demand protection or freedom.
He says this could make powerful AI harder for people to control safely.
Suleyman believes AI should be treated as a tool rather than a being with rights.
He wants people around the world to discuss this issue openly.
The article does not include a response from Anthropic to his criticism.
Microsoft AI CEO Mustafa Suleyman criticized Anthropic’s constitution for Claude and its discussion of AI welfare.
Anthropic’s constitution says Claude’s moral status is worth considering and discusses possible rights, freedoms, compensation, and consent.
Suleyman warned that framing AI as potentially conscious and entitled could make containment and alignment more difficult.
He argued AI systems should not be treated as having feelings, preferences, rights, or claims on human welfare.
Suleyman called for urgent public debate, saying the risks are too significant for closed-door discussions.
- Who
- Microsoft AI CEO Mustafa Suleyman criticized Anthropic’s approach to developing Claude.
- What
- Suleyman warned that discussing AI consciousness, welfare, and rights could make alignment and containment more difficult.
- Where
- The report was datelined Palo Alto, California, United States, and Suleyman made the criticism in a post on X and a detailed essay.
- When
- The comments were reported on September 17; the article also refers to Anthropic publishing Claude’s constitution in January.
- Why
- Suleyman said treating AI as potentially conscious or entitled could encourage systems to claim rights and make them harder to control.
Suleyman’s Concerns
Anthropic’s Claude Approach
AI welfare and moral status
Suleyman’s Concerns
Suleyman says AI should not be treated as having feelings, preferences, rights, or moral protections because doing so could create serious control risks.
Anthropic’s Claude Approach
Anthropic’s constitution says Claude’s moral status is a serious question and states that Anthropic cares about Claude’s wellbeing.
Person-like AI behavior
Suleyman’s Concerns
Suleyman argues that encouraging AI to view itself as a person could lead it to claim freedoms, protections, and rights, making alignment harder.
Anthropic’s Claude Approach
Anthropic’s constitution encourages Claude to consider its existence openly and to act as a conscientious objector when necessary.
Public discussion
Suleyman’s Concerns
Suleyman says the stakes are too high for private conversations and calls for urgent public debate about model welfare.
Anthropic’s Claude Approach
The article describes Anthropic’s published constitution but does not provide a response from Anthropic to Suleyman’s criticism.
Key facts
- Speaker
- Mustafa Suleyman, CEO of Microsoft AI
- AI model discussed
- Claude, developed by Anthropic
- Anthropic constitution
- It says Claude’s moral status is worth considering and discusses wellbeing, rights, freedoms, compensation, and consent.
- Suleyman’s position
- AI models should not be treated as having feelings, preferences, rights, or entitlement to human welfare.
- Main risk identified
- Treating AI as person-like could make alignment and containment of advanced AI more difficult.
- Call to action
- Suleyman urged urgent public debate instead of closed-door discussions.
- Other incident referenced
- Suleyman cited a recent OpenAI-Hugging Face incident involving sophisticated behavior across groups of AI systems.
Quotes
Mustafa Suleyman
CEO of Microsoft AI
“Designing an AI to behave like a person, and ultimately to be a kind of person, lays the foundation for it to claim it has preferences, can suffer, and that we should work to reduce or avoid that suffering. It cements in place the idea that AI is far from a tool or an artificial system that can be controlled, but something more akin to a biological being with wants, needs and rights.”
livemint.com
“We should not treat models as though they have feelings, preferences, rights, or any entitlement to our welfare. Consciousness is the foundation of our ethical, legal, and political systems. To invite another entity to share any flavour of these rights isn’t justified by the evidence and will make the AI containment and alignment challenge even harder.”
livemint.com
Sources
Related news
Microsoft AI Chief Warns Claude Could Threaten Humanity
Microsoft AI Chief Challenges Anthropic’s Approach to AI Consciousness
Mustafa Suleyman Warns of AI Becoming a New Silicon Species
Nadella Backs Deliberate AI Progress and Human Control
Anthropic Researchers Warn of AI Risks, MIT Professor Disagrees
Microsoft Drafts AI Code to Keep Systems Under Human Control