Archive · 14 Sep 2026
Former AI Researcher Warns Superintelligence Could Become Uncontrollable
Jacob Coxon used a scary comparison to explain advanced artificial intelligence.
He said building superintelligent AI could be like calling an alien species.
The comparison means humans might create a mind they do not understand or control.
Coxon also said some AI builders believe the technology could someday destroy humanity.
Other well-known technology leaders have raised similar concerns.
Some have discussed slowing down AI development to make it safer.
Several safety researchers have left major AI companies after becoming worried about the direction of their work.
Geoffrey Hinton has estimated that AI could carry a 10–20% risk of causing human extinction.
The people quoted in the article are calling for stronger safety measures and guardrails.
Jacob Coxon, a former OpenAI and Anthropic researcher, compared superintelligent AI to summoning an alien species.
Coxon said AI developers earnestly believe the technology could potentially wipe out humanity by the end of the decade.
Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and Elon Musk have discussed possibly slowing AI development.
Geoffrey Hinton has estimated a 10–20% chance of human extinction from AI, while Musk has cited a 10% possibility within 10 years.
Several AI safety researchers, including Jan Leike, Ilya Sutskever and Daniel Kokotajlo, have left OpenAI over concerns about safety and leadership.
- Who
- Jacob Coxon, other former AI safety researchers, and technology leaders including Dario Amodei, Sam Altman, Elon Musk and Geoffrey Hinton.
- What
- Warnings that superintelligent AI could become uncontrollable and potentially threaten humanity.
- When
- Coxon's warning was reported recently; the article also discusses departures from OpenAI in 2024 and other resignations earlier this year.
- Where
- The article does not specify where Coxon made the television appearance.
- Why
- Coxon and other researchers cited concerns about rapid AI development, weakened safety priorities and inadequate safeguards.
- Central warning
- Coxon compared developing superintelligent AI with summoning an alien species that humans may not understand or control.
- Reported extinction estimate
- Geoffrey Hinton has cited a 10–20% chance of human extinction caused by AI.
- Elon Musk's estimate
- Musk has repeatedly pointed to a 10% possibility of AI wiping out humanity within 10 years.
- Leaders discussing a slowdown
- Dario Amodei, Sam Altman and Elon Musk have spoken about the possibility of slowing AI development.
- OpenAI departures
- Jan Leike, Ilya Sutskever and Daniel Kokotajlo were among researchers who left OpenAI amid safety or governance concerns.
- Earlier Anthropic resignation
- Mrinank Sharma, formerly head of Anthropic's safeguards research team, resigned after expressing concern about AI safety and deployment pressures.
- Reported compensation dispute
- Leopold Aschenbrenner reportedly declined an equity payout worth nearly $1 million because he would not sign departure-related restrictions.