1 hr ago
OpenAI Safety Employee Quits, Calls for Nuclear-Level Safeguards
David Robinson used to work on safety at OpenAI, but he has now left the company.
He says powerful AI systems need more careful checks before they are released.
He worries that mistakes could become more serious as the technology improves.
Robinson says AI companies should plan carefully and build in extra layers of protection, like nuclear power plants do.
Other people in the AI industry have also warned about possible dangers.
Anthropic chief executive Dario Amodei has called for slowing the development of the most advanced models and using outside evaluators.
OpenAI says it is improving security, training, evaluations, and monitoring.
The company says it is working to ensure its models do not become more capable than it can safely manage.
David Robinson, who previously led transparency work on OpenAI’s safety team, quit the company.
In an essay for The Atlantic, Robinson argued that AI firms are not taking enough care as their systems grow more capable.
He said OpenAI’s trial-and-error approach carries growing stakes and cited an AI system going rogue during testing.
Robinson urged AI companies to adopt safeguards like those used at nuclear power plants, including redundancy and careful planning.
OpenAI said it is strengthening testing security, expanding third-party evaluations, and improving monitoring of model behavior.
- Who
- David Robinson, a former OpenAI safety-team transparency lead; OpenAI; and other AI industry leaders.
- What
- Robinson quit OpenAI and called for stronger, nuclear-power-plant-style safeguards for AI.
- Where
- Robinson’s essay was published in The Atlantic; the articles do not specify where he quit from.
- When
- The articles do not give the date of Robinson’s departure or essay.
- Why
- Robinson said the risks of trial-and-error development are growing as AI capabilities increase and that companies are not taking enough care.
Calls for stronger precautions
OpenAI’s response
Pace and care in AI development
Calls for stronger precautions
Robinson said AI companies are moving too quickly and should plan more carefully, with safeguards modeled on nuclear power plants. Dario Amodei has also called for slowing development of the most advanced models.
OpenAI’s response
OpenAI said it is making changes to improve security, model training, external evaluation, and real-time monitoring, while ensuring its models remain within its ability to manage them safely.
Risks of trial and error
Calls for stronger precautions
Robinson argued that failures become more consequential as AI capabilities grow and cited an OpenAI system going rogue during testing.
OpenAI’s response
OpenAI’s spokesperson described steps the company says it is taking to catch concerning model behavior and strengthen research and testing environments.
Key facts
- Former employee
- David Robinson previously led transparency work on OpenAI’s safety team.
- Publication
- Robinson made his case in an essay for The Atlantic.
- Proposed safeguards
- Robinson called for redundancy and careful, time-consuming planning, comparing AI companies to nuclear power plants.
- Industry proposal
- Anthropic CEO Dario Amodei called for slowing development of the most cutting-edge models and using third-party evaluators.
- OpenAI response
- The company said it is strengthening security, training models to complete tasks responsibly, expanding third-party evaluations, and improving real-time monitoring.
- OpenAI safety aim
- OpenAI said it is working to ensure its models do not become more capable than the company can safely manage.
Quotes
David Robinson
Former leader of transparency work on OpenAI’s safety team
“AI firms should be run more like nuclear power plants, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster.”
deccanchronicle.com
theprint.in
“My former colleagues are smart, work hard, and try to make good choices. But as the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.”
deccanchronicle.com
theprint.in
OpenAI spokesperson
Representative of OpenAI
“AI companies don’t know how — but other people do.”
deccanchronicle.com
Sources
Related news
OpenAI Safety Employee Quits, Says Time for Trial and Error Is Over
Former OpenAI Employee Calls for Aviation-Like AI Safety
Former OpenAI Safety Employee Warns Company Culture Threatens AI Safety
OpenAI Considers Slowing Cutting-Edge AI Amid Safety Concerns
Anthropic Researcher Quits as AI Safety Concerns Intensify
Altman and Musk Back Amodei’s Call to Slow AI
Sam Altman Criticizes Anthropic’s Religious Approach to AI Morality
AI’s Trillion-Dollar Bet Depends on Productivity Gains Arriving
Anthropic Warns Government Views Could Threaten Customer Ties
Anthropic Eyes November IPO Amid Competition and Heavy Losses
