Science & Tech · AI · 1 hr ago
OpenAI adds tools to detect child abuse images and restrict harmful ChatGPT use
OpenAI has introduced an image tool to detect suspected child abuse material, including images made with AI.
It is designed for child-safety groups and other vetted organizations, which can access it through OpenAI’s API at no cost.
The tool scores images so human reviewers can prioritize them, and early tests suggest it could cut images sent for review by 48% while detecting roughly the same amount of abusive material as OpenAI’s existing tool.
ChatGPT will warn and suspend users for 24 hours when they try to find abusive material, and repeat violations could lead to longer restrictions, account closure or reports to authorities.
AI-generated images can be new and lack the digital fingerprints used by some existing detection systems, making them harder to identify.
OpenAI developed the classifier with help from the National Center for Missing & Exploited Children and is working with child-safety organizations and law enforcement.
The tools aim to help detect abuse and support investigations, while OpenAI says it will keep refining them and working with outside groups.
OpenAI announced new tools on November 16 to detect child sexual abuse material, including AI-generated images, and strengthen safeguards in ChatGPT.
Its image classifier scores images for visual signs of abuse so human reviewers can prioritize cases.
OpenAI says early testing suggests the classifier could reduce images sent for review by 48% while finding roughly the same number of abusive images as its existing tool.
ChatGPT will warn users who try to find abusive material, remove the flagged conversation and suspend access for 24 hours.
OpenAI is also offering tools and guidance to vetted organizations, developers and child-safety investigators.
- Who
- OpenAI, with support from child-safety organizations and the National Center for Missing & Exploited Children.
- What
- It announced an image classifier for suspected child sexual abuse material and new ChatGPT warnings and suspensions.
- When
- November 16, 2026.
- Where
- In ChatGPT and through OpenAI’s API for vetted organizations.
- Why
- To help detect child sexual abuse material, including AI-generated content, and prevent misuse of its tools.
This story does not have two clearly opposing sides.
This story does not have a timeline yet.
- Announcement
- November 16, 2026
- Image review
- Early testing suggested 48% fewer images for review while finding roughly the same number of abusive images
- ChatGPT suspension
- 24 hours for flagged activity
- Classifier access
- No cost through OpenAI’s API for vetted organizations
- Development
- OpenAI worked with the National Center for Missing & Exploited Children
5 sources
OpenAI introduces stricter ChatGPT interventions to prevent the misuse of AI for child exploitation
OpenAI, çocuk istismarı görüntülerini tespit etmek için yeni araç geliştirdi
OpenAI launches tool for finding child sexual abuse material
OpenAI child safety tools target AI-generated abuse
OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM
What people are saying
Individual companies like OpenAI have a responsibility to prevent our technology from being exploited by predators to harm children, and we also must work together with the broader industry, child safety organizations and law enforcement to advance our collective efforts to protect children online
OpenAI is setting an important precedent for transparency in child safety
Builders need to know how well detection tools work and where they fall short.