3 hrs ago
AI Apocalypse Fears Grow as Experts Debate Humanity’s Future
Some AI experts worry that very powerful computers could become dangerous to people.
They say AI might help create cyberattacks, weapons or harmful biological materials.
AI could also spread false information or manipulate people through conversations.
Geoffrey Hinton says there may be a 10% chance that AI could kill everyone, possibly within ten years.
Another survey also found a 10% chance of very serious harm within five years if safety rules are not improved.
Other estimates are less alarming, so experts do not agree on how likely a disaster is.
The United States and China are competing to build more powerful AI systems.
Critics worry that this race could make companies prioritize speed over safety.
Experts say safer AI design and cooperation between countries could help prevent catastrophe.
AI researchers and safety experts are increasingly warning that advanced systems could cause catastrophic or existential harm.
Geoffrey Hinton estimates a 10% chance that AI could kill all humans, potentially within a decade.
A survey of more than 270 experts found a 10% chance of catastrophic harm within five years without stronger safeguards.
Potential dangers include cyberattacks, weapons development, biological threats, deception, manipulation, self-replication and misinformation.
Advocates urge the United States and China to slow AI development and improve safety, but competition between them makes that difficult.
- Who
- AI researchers, safety experts, technology companies and policymakers, including Geoffrey Hinton and leaders from OpenAI, Anthropic and Google DeepMind.
- What
- They are debating whether advanced AI could cause catastrophic or existential harm and how development should be made safer.
- Where
- The debate is global, with the United States and China identified as the two major AI development hubs.
- When
- The concerns include developments and surveys from the mid-2020s, with some forecasts covering the next five to ten years.
- Why
- Experts fear that increasingly capable AI could enable cyberattacks, weapons development, biological threats, manipulation, misinformation or other large-scale harm.
Those Warning of Existential Risk
Those Favoring Continued Development With Safeguards
Likelihood and severity of catastrophe
Those Warning of Existential Risk
Researchers including Geoffrey Hinton and former Anthropic researcher Jacob Coxon warn that advanced AI could potentially cause mass death or even end humanity.
Those Favoring Continued Development With Safeguards
Not all estimates are equally alarming; the cited expert survey measured a 10% chance of catastrophic harm, rather than specifically predicting human extinction.
Speed of AI development
Those Warning of Existential Risk
Safety advocates argue that the United States, China and AI companies should slow development because competition may make safety secondary to capability gains.
Those Favoring Continued Development With Safeguards
The article indicates that companies and nations continue pursuing more capable systems because they are competing to shape the future of AI.
How to prevent disaster
Those Warning of Existential Risk
Advocates call for stronger safeguards, international coordination and an agreement between the United States and China on the pace and direction of development.
Those Favoring Continued Development With Safeguards
Geoffrey Hinton argues that a key solution is designing AI so it does not seek to eliminate humans, suggesting safer system design as a central remedy.
Key facts
- Geoffrey Hinton’s estimate
- A 10% chance that AI could kill all humans, potentially within a decade.
- MIT AI Risk Initiative survey
- More than 270 international AI experts assessed a 10% chance of catastrophic harm within five years without additional safeguards.
- Definition of catastrophic harm
- More than one million human deaths or over $100 billion in financial losses.
- 2023 industry statement
- AI leaders called for mitigating extinction risks from AI alongside risks such as pandemics and nuclear war.
- American concern
- More than 50% of Americans surveyed in 2024 were concerned about AI’s existential threats.
- Support for regulation
- An AI Policy Institute survey found that 83% of Americans favored regulating or slowing AI development.
- Main development hubs
- The United States and China are described as being locked in competition to build more capable AI systems.
Quotes
Geoffrey Hinton
AI researcher and Nobel Prize winner discussing the estimated risk of AI killing all humans
“It will not only be able to cause complete chaos just by talking to people, but also it will be able to design nasty computers and biological viruses”
NDTV
“It doesn't seem unreasonable to me”
NDTV
Jacob Coxon
Researcher who left Anthropic and expressed concern about AI risks
“No other human activity poses this level of danger”
NDTV
Sources
Related news
AI Could Threaten Humanity, Godfather of AI Warns
AI Experts Warn Advanced Systems Could Threaten Humanity
Former Anthropic Researcher Warns AI Could Kill Humanity This Decade
Anthropic Researcher Quits as AI Safety Concerns Intensify
Nvidia CEO rejects AI extinction fears as safety warnings intensify
Former Anthropic Researcher Warns AI Could Outpace Nuclear Threats
OpenAI Considers Slowing Frontier AI as Safety Concerns Grow
Silicon Valley Keeps Moving the Goalposts on AGI Claims
AI Reshapes Patient-Centric Healthcare and Life Sciences Progress
