A look at AI safety debates and the rising popularity of "Stop Unilaterally + Raise Awareness", or SURA; safety-focused AI labs shouldn't abandon the AI race

Your coding assistant's capabilities may soon be directly tied to its alignment research progress, not just its raw power.

Scott Alexander's recent analysis on LessWrong argues against the "Stop Unilaterally + Raise Awareness" (SURA) approach to AI safety, advocating instead for a more pragmatic strategy. Alexander categorizes AI transition plans into Plan A (a global, decades-long pause for research and fortification) and Plan C (a race dynamic where alignment-focused labs maintain a lead and "burn" it on alignment research before releasing superintelligence). He contends that the recent surge of interest in SURA, which advocates for unilateral cessation of AI development and public awareness campaigns, overlooks the critical importance of Plan C. SURA proponents believe alignment is doomed and focus solely on Plan A, potentially jeopardizing the only viable fallback if a pause fails. Alexander critiques SURA for potentially handing the lead to non-alignment-focused labs, reducing the effectiveness of alignment research, and creating a worse outcome in a continued race scenario. He argues that while the chances of success for Plan C are slim, abandoning it entirely is a worse strategy than optimizing for it as a backup.

The debate highlights a fundamental split in AI safety philosophy: prioritizing an unlikely global pause (Plan A) versus a more complex, riskier race-for-safety strategy (Plan C). SURA's popularity indicates a growing sentiment that achieving alignment is exceptionally difficult, leading proponents to advocate for radical, unilateral halts in development rather than managing risks within an ongoing race. This perspective suggests a potential shift away from incremental improvements and towards drastic measures, even if they carry their own significant risks, such as ceding ground to less safety-conscious actors.

Alexander's analysis suggests that if the dominant strategy within AI development shifts towards SURA's principles, the practical implications for future AI systems could be profound. Instead of optimizing AIs for maximum capability and then attempting to align them, future development might explicitly factor in and prioritize alignment research as a core component, potentially limiting raw performance gains in favor of safety assurances. This could lead to AIs that are demonstrably safer but potentially less advanced in certain capabilities compared to those developed under a pure race dynamic.