Archive · 9 Nov 2025
OpenAI Warns of Catastrophic AI Risks, Proposes Safety Measures
Imagine computers are getting super smart, like, smarter than even the best human brains in some ways.
OpenAI, the company that made ChatGPT, is saying these super-smart computers, called AI, could do amazing things and help us discover new science by 2026 and even bigger things by 2028.
But, they also said that these super-smart AIs could be very dangerous if we don't figure out how to control them safely.
It's like having a powerful tool that could be used for good or bad.
OpenAI thinks we need to do more research to make sure AI stays safe and helpful for everyone.
They also suggested that maybe the companies building AI should slow down a bit to make sure they understand all the risks.
They want governments and companies to work together to create rules, like safety instructions, to prevent bad things from happening and to make sure AI helps create a world where everyone has enough.
OpenAI warns that superintelligent AI systems pose 'potentially catastrophic' risks.
The company suggests that the AI industry may need to slow development to study safety and alignment.
OpenAI predicts AI systems could make small scientific discoveries by 2026 and major breakthroughs by 2028.
Concerns exist regarding 'recursive self-improvement' and the potential for AI to surpass human intelligence (AGI).
OpenAI advocates for global cooperation, information sharing, and unified regulation to mitigate AI risks.
- Who
- OpenAI
- What
- Warned about the potentially catastrophic risks of superintelligent AI and proposed global safety measures and research recommendations.
- When
- November 6 (blog post publication)
- Where
- Publicly, via a blog post.
- Why
- To address concerns about advanced AI capabilities like recursive self-improvement and to outline a path for safe development and deployment.
Optimistic Outlook on AI Progress
Concerns and Cautions Regarding AI Advancement
Timeline for Advanced AI Capabilities
Optimistic Outlook on AI Progress
OpenAI predicts AI systems will make 'very small' scientific discoveries by 2026 and significant discoveries by 2028, indicating rapid progress towards advanced capabilities.
Concerns and Cautions Regarding AI Advancement
AI research scientist Andrej Karpathy suggests AGI might still be a decade away, citing current limitations in areas like continual learning and cognitive abilities.
Potential Societal and Economic Impact
Optimistic Outlook on AI Progress
OpenAI suggests that a future with widely distributed abundance, driven by AI, could lead to significantly improved lives.
Concerns and Cautions Regarding AI Advancement
OpenAI acknowledges that the economic transition due to AI could be difficult and may necessitate a fundamental change in the socioeconomic contract.
Nature of AI Risks
Optimistic Outlook on AI Progress
OpenAI focuses on the potential benefits of superintelligent systems while acknowledging the need for robust alignment and control mechanisms.
Concerns and Cautions Regarding AI Advancement
OpenAI warns that superintelligent systems carry 'potentially catastrophic' risks, including those related to recursive self-improvement and misuse, emphasizing the need for deep study and caution.
- Company
- OpenAI
- Stated Risk Level
- Potentially catastrophic
- Key Concern
- Superintelligent AI systems, recursive self-improvement
- Proposed Research
- AI safety and alignment studies, considering slowing development
- Predicted Discovery Timeline
- Small discoveries by 2026, significant discoveries by 2028
- Regulation Approach
- Close collaboration with governments, unified regulation, minimal burden on current tech and open-source models
- Broader Call for Action
- Concerns raised by various public figures calling for a ban on AI superintelligence
Sources
OpenAI warns of catastrophic risk amid exponential AI development: Here's why
AI to make more ‘significant discoveries’ by 2028 and beyond, says OpenAI
‘We’ve crossed a major threshold’: Sam Altman predicts AI will soon outthink human researchers
