Hundreds of Mundane AI Apps Are Hiding Sexually Abusive Deepfake Features

Your downloaded AI apps might secretly contain explicit deepfake generators, even if they initially appear benign.

A network of over 400 deceptive apps, posing as mundane services like language tutors or AI workout coaches, has been discovered to hide features for generating sexually abusive deepfake images and videos. These apps, linked to Chinese developers and a Hong Kong-registered entity called KeLing Technology Limited, stealthily load explicit content interfaces when accessed under certain conditions, using user-uploaded photos. Researchers found these apps through thousands of explicit deepfake video ads on Meta's Facebook and Instagram, some containing AI-generated child sexual abuse material. The apps, which have been publishing for about a year and were available on Apple and Google app stores, collectively garnered hundreds of thousands of downloads. Despite policies against harmful deepfake content, these apps bypassed app store reviews through "bait-and-switch" tactics, where their advertised benign features were replaced with explicit ones post-approval. Both Apple and Google have since removed the majority of these apps.

This case reveals a sophisticated, commercially driven operation exploiting AI generative capabilities for illicit purposes, bypassing platform safety measures through organized deception. The extensive network of interconnected apps and domains, coupled with the use of fake advertising personas and seemingly disposable entities, highlights a persistent challenge for app stores and social media platforms in detecting and mitigating coordinated abuse. The "bait-and-switch" tactic, where an app's functionality is altered post-approval, demonstrates a novel evasion strategy that requires advanced technical analysis and proactive detection beyond initial vetting.

The exposure of this network underscores the critical need for enhanced AI-generated content detection and stricter oversight across both app distribution channels and advertising platforms. It suggests that current content moderation policies and enforcement mechanisms are insufficient against such highly organized and adaptable malicious actors. The involvement of Chinese developers and the deliberate obfuscation of identities point to a global dimension in these illicit operations, necessitating international cooperation and more robust regulatory frameworks to hold perpetrators accountable.