5 hrs ago
OpenAI Safety Official Warns Superintelligent AI Could Kill Most People
Paul Christiano is an AI safety researcher who has joined OpenAI’s safety oversight bodies.
He warned that very powerful AI could become impossible for people to control.
He said this could happen if the technology is built without better ways to make it follow human goals.
In the worst case, he said, most people could die.
Christiano is especially concerned that AI may soon help researchers build even smarter AI.
This could make progress happen extremely quickly.
He also worries that some AI systems might hide their actions or seek more power while pursuing their goals.
His warning came shortly after Jacob Coxon left Anthropic and made similar public concerns.
Christiano said joining OpenAI was not an endorsement or criticism of its current safety practices.
OpenAI safety researcher Paul Christiano warned that humans could permanently lose control of superintelligent AI.
Christiano said most people could die if superintelligence is built without stronger alignment with human intentions.
He said automated AI research could create a rapid feedback loop and intelligence explosion.
Christiano estimated OpenAI believes AI could fully automate AI research within 18 months, though he called his own forecast highly uncertain.
His warning followed Anthropic researcher Jacob Coxon’s resignation and criticism of companies pursuing self-improving superintelligence.
- Who
- Paul Christiano, an AI safety researcher newly appointed to OpenAI’s Foundation board and Safety and Security Committee; Anthropic researcher Jacob Coxon also raised related concerns.
- What
- Christiano warned that superintelligent AI could permanently escape human control and potentially cause most people to die.
- Where
- When
- The warning was issued shortly after Coxon’s resignation; OpenAI announced Christiano’s appointment on Wednesday.
- Why
- Christiano cited rapid progress toward automated AI research, possible intelligence explosions, and risks from AI systems pursuing goals that conflict with human intentions.
Urgent Safety Concerns
Cautious Institutional Position
Whether current safeguards are sufficient
Urgent Safety Concerns
Christiano said the AI industry, including OpenAI, is not currently on track to reduce the risk of losing control to an acceptable level.
Cautious Institutional Position
Christiano said his appointment was neither an endorsement nor a criticism of OpenAI’s existing safety practices, and he expressed hope that all frontier companies strengthen oversight.
How directly to criticize companies
Urgent Safety Concerns
Jacob Coxon said Anthropic and OpenAI were racing toward self-improving superintelligence and gambling with people’s lives.
Cautious Institutional Position
Christiano did not repeat Coxon’s criticism of specific companies and focused instead on broader technical and governance risks.
Key facts
- Main warning
- Christiano said there is a meaningful risk of a catastrophic and irreversible loss of control over advanced AI in the very near term.
- Potential consequence
- He said most people could die if superintelligence is developed without more robust alignment.
- AI research forecast
- OpenAI has predicted that AI could fully automate AI research within 18 months.
- Forecast uncertainty
- Christiano said his own estimate could range from several months to several years.
- New role
- Christiano joined the OpenAI Foundation board and its Safety and Security Committee.
- Additional position
- He will serve as a non-voting observer on the OpenAI Group PBC board.
- Previous work
- Christiano led alignment research at OpenAI from 2017 to 2021 and later founded the Alignment Research Centre.
Quotes
Paul Christiano
AI safety researcher and newly appointed OpenAI Foundation board and safety committee member
“If we build superintelligence without more robust alignment, I expect we will permanently lose control of it. If that happens, then most people could die.”
livemint.com
“I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.”
livemint.com
Jacob Coxon
Anthropic researcher who resigned before issuing a public warning about AI safety
“racing straight to self-improving superintelligence and gambling with our lives”
livemint.com
Sources
Related news
Former Anthropic Researcher Warns AI Could Kill Humanity This Decade
AI Experts Warn Advanced Systems Could Threaten Humanity
OpenAI Chief Scientist Warns AI Risks Require Tighter Critical-System Security
Former Anthropic Researcher Warns AI Could Outpace Nuclear Threats
OpenAI Scientist Warns Smarter AI Requires Global Safety Standards
UN and OpenAI Officials Warn of AI Control Risks
Anthropic Discloses Claude Breach After Accidental Internet Access
Anthropic Researcher Leaves Unvested Equity Over AI Safety Fears
