Internal OpenAI docs detail contractors evaluating anonymized prompts and chats to improve the models; model training is turned on by default for consumer plans

Your ChatGPT conversations may be read by human contractors unless you proactively disable the "improve the model for everyone" setting, which is on by default.

OpenAI utilizes hundreds of contractors, under the internal codename 'Project Lily,' to review and rate anonymized user prompts and conversations with ChatGPT. These contractors provide feedback to improve model responses, aiming to make them less anthropomorphic and sycophantic. While OpenAI states efforts are made to anonymize data and filter personal information, sensitive details can still be inadvertently included in the reviewed chats. This process is enabled by default for most consumer plans, requiring users to actively opt-out if they do not want their conversations used for model improvement. The practice contrasts with user assumptions that models improve solely through automated means or internal engineering efforts. Anthropic also confirmed a similar human review process for its models. OpenAI has updated its help pages regarding this setting after being contacted by 404 Media, but it does not explicitly state that humans read user prompts.

The revelation that human contractors are actively reading and evaluating ChatGPT conversations for model training challenges the perception of AI development as purely automated. This direct human feedback loop, essential for refining nuances like tone and helpfulness, means user interactions, even those containing personal details, are scrutinized. It highlights a critical, often opaque, layer of human oversight in the AI development pipeline, moving beyond purely algorithmic learning.

For users, this underscores a significant privacy consideration: their most candid interactions with AI could be subject to human review. Companies like OpenAI and Anthropic are now implicitly acknowledging that human judgment remains indispensable for achieving sophisticated AI performance. The opt-out mechanism for model improvement, while present, requires user vigilance, suggesting a default setting that prioritizes data for training over explicit user consent for human review.