Inside ‘Project Lily’: The Humans Reading Your ChatGPT Chats

Your ChatGPT conversations might be read by human contractors, even if you believe they are private.

OpenAI is employing hundreds of contractors globally to read and analyze user prompts submitted to ChatGPT. These human reviewers, working under the codename 'Project Lily,' are tasked with rating and critiquing the AI's responses to improve its performance. The prompts reviewed can include sensitive personal information, as users often confide in the chatbot about their lives, seeking advice or companionship. While OpenAI states it attempts to anonymize prompts and uses a privacy filter, internal documents indicate that sensitive details can still be exposed. Contractors are trained to identify and escalate prompts with safety concerns or personal information. This process, distinct from safety flagging, is a core component of model improvement, alongside internet scraping and AI development. Anthropic has confirmed a similar human review process for its models. OpenAI's 'improve the model for everyone' setting, which is on by default for most users, enables this data usage, though enterprise clients have it off by default.

The reliance on human contractors to review user prompts highlights a crucial, often unseen, aspect of AI development. It moves beyond algorithmic training and internet data, revealing the significant labor involved in refining large language models. This practice raises privacy concerns, particularly as users may not be fully aware their personal conversations are being processed by external reviewers, despite default settings that allow this data use for model improvement.

This disclosure challenges the perception of AI as a purely automated system and underscores the human element in its evolution. It suggests that companies like OpenAI and Anthropic are prioritizing nuanced feedback on AI tone, helpfulness, and trustworthiness, moving past simple accuracy. Future AI development will likely involve more sophisticated methods for anonymizing user data while still leveraging human insight for refinement, balancing performance gains with user privacy expectations.