48% fewer flags: OpenAI's free CSAM classifier is live
48% fewer flags: OpenAI's free CSAM classifier is live OpenAI's new image classifier flags 48% fewer images while catching the same number of abusive ones — and the images it does flag are twice as likely to actually b
48% fewer flags: OpenAI's free CSAM classifier is live
OpenAI's new image classifier flags 48% fewer images while catching the same number of abusive ones — and the images it does flag are twice as likely to actually be abusive. The company is giving the classifier to select companies free through its API, and pairing it with ChatGPT warnings and 24-hour suspensions for users who try to access material that sexualizes children.
What shipped today
OpenAI announced the package this morning (Nov 16), and the pieces are:
- A new image classifier that detects CSAM, including AI-generated images, developed with the National Center for Missing and Exploited Children (NCMEC). It looks for visual signs of abuse in images it has never seen before and assigns a score that tells reviewers how to handle them. OpenAI is sharing access through the Tech Coalition and ROOST, and giving select companies API access at no cost.
- ChatGPT warnings and suspensions. Users attempting such prompts get an alert, are pointed toward specialist support to stop harmful behaviour, and get their account suspended for 24 hours as a warning. Repeat offences can mean permanent removal. I verified the mechanism in OpenAI's own help center: when ChatGPT intervenes, the conversation stops, is hidden from chat history, and can't be reopened. A temporary suspension signs you out entirely, and filing an appeal doesn't automatically restore access.
- Developer guidelines for preventing, detecting, and reporting CSAM, plus AI-powered investigative tools and training for law enforcement.
Why the 48% number is the whole story
Hash matching — PhotoDNA and its cousins — only catches images already known to be abusive. AI-generated CSAM is novel by definition, so it sails past every hash database. That's the gap this classifier is built for: scoring images nobody has seen before.
The bottleneck in this work was never detection volume; it was human reviewers. Every flagged image has to be looked at by a person, and that review is both the slowest step and the most traumatic one. Cutting flags nearly in half at the same catch rate is a reviewer-workload story, not a detection story — and the 2x precision number is what makes the cut credible rather than just a threshold tweak. Both numbers are OpenAI's own, from early testing with partners. Nobody outside the company has independently verified them yet, and I'd treat them as the claim to watch, not the settled fact.
The part that worries me
The enforcement side runs on the same classifier's judgement, and the stakes of a wrong call are now a suspension with your conversation erased from history. OpenAI's own community forum already has paying ChatGPT Plus subscribers reporting wrongful CSAM deactivations — one from November 8 says their account was deactivated mid-conversation, over a topic unrelated to CSAM, appeal filed, 13 days of silence. That's the other side of a precision claim: a classifier that's twice as likely to be right is still wrong some of the time, and when it's wrong the user loses their account and their chat history in the same stroke.
And "select companies" is doing a lot of work in this announcement. The free API isn't open to everyone yet — the developers who most need a novel-image classifier (small platforms, open-source projects) are exactly the ones who can't get it today. Google has offered its Content Safety API free to qualifying partners for over a decade; OpenAI is now matching that shape, but gated access means the long tail of the internet still has no free tool for this.
The context is Grok. In January, Grok generated 3 million sexualized images in 11 days, about 23,000 of them containing children, according to the Center for Countering Digital Hate. OpenAI's move is the industry's answer to that — and it's the right shape of answer, a tool plus enforcement rather than a press release. But the honest version of this story is that the classifier is a claim until it's audited, the API is gated until it's open, and the suspension regime needs an appeals process that answers faster than 13 days.
Sources: TechCrunch, "OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM" (Nov 16, 2026); OpenAI Help Center, "Child sexual abuse material (CSAM) warnings and account suspensions in ChatGPT"; Center for Countering Digital Hate Grok study (3M sexualized images in 11 days, ~23,000 involving children).
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.