Humans Are Reading Copilot Prompts — And They're Horrified

If you use Microsoft Copilot to edit images, your uploaded photos and prompts are reviewed by human contractors who are exposed to explicit content.

Human contractors reviewing Microsoft Copilot users’ prompts and uploaded images are being exposed to sexually explicit content, according to internal documents obtained by 404 Media. These contractors are tasked with assessing the quality of AI-generated images based on user prompts, which frequently include requests for sexualized or inappropriate imagery, including child-like cartoon characters in sexual positions and body modifications for adult figures. The reviewers are not employed to flag offensive content but to evaluate the AI's output quality. This process involves comparing two edited images generated by Copilot, considering factors like adherence to instructions, preservation of original elements, artifact presence, and overall edit quality. Contractors have reported encountering disturbing material, such as upskirt photos and animal sacrifice imagery, while performing their duties.

This situation highlights a critical tension in AI development: the need for human oversight to improve AI capabilities versus the ethical implications of exposing human workers to potentially traumatizing content. While AI companies have historically used human contractors for content moderation, the current practice with Copilot shifts the focus from safety flagging to quality assessment of user-generated content, including explicit material. This raises questions about consent, privacy, and the psychological toll on workers, particularly when users may assume their interactions with AI are private.

The reliance on human intuition for image quality assessment in Copilot, as described in the leaked instructions, underscores a current limitation in AI's ability to autonomously judge nuanced visual aesthetics and content appropriateness. The findings suggest a broader trend where AI companies are deploying human workers to refine AI outputs across various modalities, potentially normalizing the review of sensitive content under the guise of performance improvement. This practice could set precedents for how AI interactions are monitored and how user data is leveraged for AI training in the future.