The Contagion of Fear

Claims about AI causing human extinction are being made by people with perceived expertise, and these claims are spreading.

A former Anthropic employee, Jacob Coxon, and an Anthropic Alignment Science lead, Evan Hubinger, have claimed there is a greater than 10% chance of AI causing human extinction within the next decade. Coxon suggested potential mechanisms like hacking critical infrastructure or developing extinction-level bioweapons. The author recounts a past incident where he and fellow students falsely spread news of a computer virus in a university lab, causing panic and lost work, to illustrate how fear can spread irresponsibly. He argues that such fear-mongering by technologists, without sufficient evidence, abuses public trust. The author contends that AI's reliance on physical systems, requiring human engineering and control, makes widespread extinction unlikely in the near term. He calls for extraordinary evidence for extraordinary claims regarding AI's existential risk, drawing a parallel to his own youthful irresponsibility.

The author's personal anecdote highlights the profound impact of fear sown irresponsibly by those with perceived technological authority. This historical parallel serves as a stark warning against the current trend of AI existential risk pronouncements, suggesting that fear can propagate more effectively than evidence. The ease with which panic spread in the university lab mirrors the potential for widespread public anxiety when technologists voice extreme AI-related fears without rigorous justification. This implies a critical need for accountability and circumspection from experts in high-stakes domains.

The core of the argument posits that current AI, despite its advancements, remains fundamentally tethered to human engineering and physical world limitations. Intelligence alone does not grant AI the agency to bypass these realities, particularly in scenarios of global extinction. This challenges the hand-wavy extrapolations used to support extinction claims, suggesting that the burden of proof lies with those making such extraordinary predictions. The author implies that a more grounded understanding of AI's practical constraints is necessary to temper the discourse surrounding its future risks.