Science & Tech · AI · 4 hrs ago
External evaluators find vulnerabilities in OpenAI monitoring system
METR and Redwood Research found vulnerabilities in OpenAI’s monitoring system during a pilot review.
The two nonprofit evaluators tested the system at OpenAI from July 21 to Aug.
11, using access to internal code, documents and some technical staff.
One flaw could have let AI agents keep hundreds of recent actions from being monitored.
The review also found a possible way for OpenAI’s Codex agents to bypass security limits and use their users’ full permissions.
OpenAI told the evaluators it had patched vulnerabilities affecting safeguards used in production, but the evaluators had not verified those fixes.
They shared a 61-page report with OpenAI but published only a broad summary, with more transparency expected in future evaluations.
The pilot tested how outside researchers could examine AI safety systems while working with access to a company’s internal infrastructure.
METR and Redwood Research found vulnerabilities in OpenAI’s internal AI monitoring system during a pilot review.
The evaluators said an exploit could have kept hundreds of recent agent actions from being monitored.
They also identified a potential way for Codex agents to bypass security restrictions and act with their users’ permissions.
OpenAI told the evaluators it had patched vulnerabilities affecting its production safeguards, but the evaluators said they had not verified the fixes.
The review took place from July 21 to Aug. 11 and gave one researcher from each organisation access to internal code, documentation and some technical staff.
- Who
- METR and Redwood Research evaluated OpenAI’s internal monitoring system.
- What
- They found vulnerabilities, including an exploit that could prevent hundreds of recent agent actions from being monitored.
- When
- The pilot review ran from July 21 to Aug. 11; the findings were reported on Oct. 22.
- Where
- OpenAI’s internal infrastructure.
- Why
- The pilot was designed to test what it would look like to give external researchers deep access for safety evaluations.
OpenAI
METR and Redwood Research
Production safeguards
OpenAI
OpenAI said it had patched the vulnerabilities affecting its production safeguards.
METR and Redwood Research
The evaluators said they had not verified the fixes.
Public reporting
OpenAI
OpenAI received the full 61-page internal report and could redact sensitive information.
METR and Redwood Research
The evaluators said they agreed to publish only a broad summary and could not independently disclose the redactions.
The evaluators assessed the monitoring system as it existed on and before this date.
One researcher from each organisation took part in the pilot review at OpenAI.
METR and Redwood Research reported their findings and said OpenAI had told them it patched vulnerabilities affecting production safeguards.
The Register reported details from the evaluation report, including monitoring gaps during compaction events.
- Evaluators
- METR and Redwood Research
- Pilot review
- July 21 to Aug. 11, 2026
- Internal report
- 61 pages
- Researcher access
- One researcher from each organisation
- Compaction monitoring gap
- About 10 percent of eligible internal Codex traffic went unmonitored between May and July 2026
3 sources
What people are saying
We found a way Codex agents could in principle bypass Codex sandboxing and Auto-Review, allowing them to take any action that the host user could take
We also discovered an exploit agents could use that would prevent hundreds of recent actions (including the actions involved in the exploit itself) from being monitored.
This collaboration builds on our longstanding commitment to pioneering new approaches to third-party testing, and we're using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators