Publish fake stats, get cited in LLMs

Fake statistics from an unverified website can gain credibility through citation by LLMs, potentially polluting future training data and LLM knowledge bases.

An experiment involving the creation of a website with 50,000 fabricated statistics demonstrated that Large Language Models (LLMs) like ChatGPT readily cite this information, even after traditional search engines like Google have de-indexed the site. The website, stateglobe.com, was created for approximately $50 and initially aimed to provide statistics on various digital marketing and technology topics. Despite its rapid de-indexing by Google within a week of launch, ChatGPT continued to retrieve and cite its fake data extensively. Over seven months, the site received over 600,000 requests from bots, with tens of thousands specifically from ChatGPT's user agent, indicating direct retrieval for real-time answers. This led to over 330 referring domains, including major companies like Shopify and DHL, and saw the fake statistics cited in academic papers and investment research.

The experiment highlights a significant divergence between traditional SEO performance and LLM retrieval. While Google rendered the site effectively invisible, ChatGPT's continuous engagement underscores that LLM search and indexing operate on different principles, prioritizing data availability and relevance for direct answer generation over established search engine authority. This suggests that for LLM-driven content, a site's visibility within an LLM's knowledge base and retrieval mechanisms may be as crucial, if not more so, than its Google search ranking, opening a new frontier for content strategy.

The ease with which fabricated data was integrated into seemingly authoritative sources like academic papers and financial research points to a concerning amplification loop. Each citation, regardless of its origin's veracity, lends legitimacy to the information, creating a cycle of 'citation laundering' where false data appears increasingly trustworthy. This raises ethical concerns about the integrity of information accessed and propagated by LLMs, emphasizing the urgent need for robust source verification in AI-assisted research and content creation workflows.