OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM
First reported by TechCrunch ·
Your account may be suspended for 24 hours if you attempt to access CSAM content through ChatGPT.
OpenAI has introduced new measures to combat child sexual abuse material (CSAM) across its AI products. These include a novel image classifier, developed with the National Center for Missing and Exploited Children (NCMEC), which can detect CSAM in images, including AI-generated ones. This tool has shown in early tests to reduce the workload for human reviewers by flagging fewer non-abusive images while maintaining detection rates. Additionally, ChatGPT will now feature warning screens for users attempting to access or generate content that sexualizes children. These warnings will direct users to support resources and may result in temporary 24-hour account suspensions for first-time offenses, with repeat violations leading to permanent bans. OpenAI is also providing enhanced AI tools, training, and a field guide for law enforcement agencies investigating child safety cases. New guidelines have also been released for developers using OpenAI's tools to help them implement child safety measures and report CSAM.
OpenAI's proactive stance with its new CSAM detection tools signals a broader industry trend towards AI companies taking greater responsibility for the misuse of their technology. The free API access to the image classifier for select companies suggests a strategy to foster industry-wide adoption of safety standards, potentially setting a precedent for how AI safety research is shared and implemented. This move also addresses mounting legal and public pressure on AI developers to mitigate risks, particularly concerning the generation of harmful content.
The enhanced capabilities for law enforcement, coupled with developer guidelines, indicate a concerted effort to create a more robust ecosystem for child protection against AI-enabled threats. By integrating safety features directly into platforms like ChatGPT and providing specialized tools for investigations, OpenAI is attempting to move beyond reactive content moderation to proactive prevention and enforcement. Future developments will likely focus on the efficacy of these tools in real-world scenarios and the broader adoption of similar safety measures by other AI providers.
AI-written summary. May contain errors.