Science & Tech · AI · 1 hr ago

OpenAI adds tools to detect child abuse images and restrict harmful ChatGPT use

OpenAI adds tools to detect child abuse images and restrict harmful ChatGPT use

OpenAI has introduced an image tool to detect suspected child abuse material, including images made with AI.

It is designed for child-safety groups and other vetted organizations, which can access it through OpenAI’s API at no cost.

The tool scores images so human reviewers can prioritize them, and early tests suggest it could cut images sent for review by 48% while detecting roughly the same amount of abusive material as OpenAI’s existing tool.

ChatGPT will warn and suspend users for 24 hours when they try to find abusive material, and repeat violations could lead to longer restrictions, account closure or reports to authorities.

AI-generated images can be new and lack the digital fingerprints used by some existing detection systems, making them harder to identify.

OpenAI developed the classifier with help from the National Center for Missing & Exploited Children and is working with child-safety organizations and law enforcement.

The tools aim to help detect abuse and support investigations, while OpenAI says it will keep refining them and working with outside groups.

5 sources

Related

What people are saying

OpenAICompany statement
Individual companies like OpenAI have a responsibility to prevent our technology from being exploited by predators to harm children, and we also must work together with the broader industry, child safety organizations and law enforcement to advance our collective efforts to protect children online
via Mashable
Camille FrançoisPresident of ROOST
OpenAI is setting an important precedent for transparency in child safety
via Mashable
Builders need to know how well detection tools work and where they fall short.
via Mashable