1 day ago
Former OpenAI Researchers Urge Oversight of AI Models
Three people who used to work at OpenAI wrote a letter about keeping AI systems safe.
They said companies should be able to inspect how AI works through a process called chain-of-thought.
This is a written record of the steps an AI takes when answering a question.
The record is not a perfect way to know what an AI will do, but the former employees say it can still help researchers spot problems.
They also want companies to work more with outside safety experts.
OpenAI said it agrees with the letter’s recommendations.
The company said the three people were dismissed for policy violations, not for raising safety concerns.
The letter’s authors said they had not shared information outside their job duties.
Three former OpenAI employees urged the company to preserve the ability to oversee AI models’ chain-of-thought.
The letter said companies should not pursue developments that further reduce oversight of models.
The former employees also called for closer collaboration with external security auditors.
OpenAI said it agreed with the recommendations and that the dismissals were not due to safety concerns or speaking out.
The letter described chain-of-thought oversight as useful but imperfect, and cited research showing promise for detecting improper AI behaviour.
- Who
- Former OpenAI employees Jasmine Wang, Tomek Korbak, and Mikita Balesni, and OpenAI.
- What
- The former employees urged OpenAI to maintain oversight of AI models’ chain-of-thought and work more closely with external security auditors.
- Where
- The letter was addressed to OpenAI’s board members and safety committees.
- When
- The letter followed the employees’ dismissals the previous week; the article does not give a date.
- Why
- The former employees said oversight helps researchers understand powerful AI systems and could help reduce safety risks.
Former employees’ position
OpenAI’s position
Oversight and safety
Former employees’ position
The former employees warned that reducing the ability to oversee AI models could pose safety risks and urged companies to preserve chain-of-thought oversight.
OpenAI’s position
OpenAI said the ability to oversee models is of paramount importance and that it agrees with the letter’s recommendations.
Reason for dismissals
Former employees’ position
The former employees said they did not believe they had interacted with external parties outside the scope of their job duties.
OpenAI’s position
OpenAI said the three were dismissed for violating policies on access to and handling of confidential information, not for raising safety concerns or speaking out.
Key facts
- Letter writers
- Jasmine Wang, Tomek Korbak, and Mikita Balesni
- Former employer
- OpenAI
- Main recommendation
- Preserve the ability to oversee AI models’ chain-of-thought
- Additional recommendation
- Collaborate more closely with external security auditors
- OpenAI’s response
- The company said it agreed with the recommendations and that the dismissals were not due to raising safety concerns or speaking out.
- Chain-of-thought
- A written record of how an AI model processes a query, including its problem-solving steps
- Research cited
- A paper co-authored by Korbak found the oversight method imperfect and potentially fragile, but promising for detecting improper AI behaviour.
Quotes
The three former OpenAI employees
Former OpenAI employees who worked on safety and alignment research
“As an industry, we do not yet know how to safely develop and deploy models that we cannot oversee.”
thehansindia.com
“We do not fire employees for raising concerns.”
thehansindia.com
Sources
Related news
Fired OpenAI Researchers Challenge Dismissals and Warn of Safety Risks
Former OpenAI Researchers Say They Were Fired Over Safety Concerns
Fired OpenAI Researchers Urge Visible AI Reasoning and Independent Safety Reviews
Former OpenAI Researchers Urge Independent Audits After Dismissals
OpenAI Parts Ways With Three Researchers Over Confidential Data
OpenAI Safety Employee Quits, Says Time for Trial and Error Is Over
Google Opens SynthID Detector Worldwide to Check AI Content
AI Development Keeps Racing Despite Calls for Safety
Anthropic Consults Religious Leaders on Claude’s Values and Consciousness
Global AI Governance Splits as Nations Debate Voluntary Safeguards
