OpenAI launches free classifier to detect child sexual abuse material, including AI-generated images

OpenAI has launched a free image classifier to help vetted organisations detect child sexual abuse material, including images generated by artificial intelligence. The announcement was made on November 16 at a press briefing in Washington, and the classifier is already available through the company's API.

The system examines visual signs of abuse in images and assigns a score to each one, helping human reviewers prioritise the most likely cases. According to OpenAI, it is the first broadly available classifier with a specific signal indicating whether material may have been AI-generated — which makes it possible to flag novel images that appear in no known database.

Built with NCMEC and shared with vetted organisations

The classifier was trained, evaluated and developed with assistance from the National Center for Missing & Exploited Children (NCMEC), the nonprofit that runs CyberTipline, the US clearinghouse for online child exploitation reports. Access is free and shared through the Tech Coalition and ROOST, and the system also supports Thorn's Safer.

In internal evaluations and with early-access partners, OpenAI reported 48% fewer images flagged for review while identifying roughly the same number of abusive images — and images flagged by the new classifier were about twice as likely to contain abuse. Partners including fal, invideo AI and SafetyKit reported their own results, with fal confirming 72% of high-confidence flags as novel material.

ChatGPT warnings and guidance for developers

In ChatGPT, attempts to find abusive material will now trigger a warning explaining the harm caused, with directions to specialist help. The flagged conversation is removed and account access is paused for 24 hours; repeated violations may lead to longer restrictions, account closure or reports to the authorities.

OpenAI also published guidance for developers on November 15 on preventing, detecting and responding to suspected cases, plus AI tools and training for specialist child-safety law enforcement units.

The announcement comes as reports keep climbing: according to NCMEC, reports of AI-generated material received by CyberTipline reached 400,000 in 2025, up from 67,000 the year before.

Sources

Featured image: OpenAI logo, via Wikimedia Commons.

This article was produced with the assistance of artificial intelligence, cross-checked against public posts on X and internet sources. Consult the original sources to confirm the information.

By Muse