Anthropic CEO says it’s time to pump the brakes on AI

If you are developing or using advanced AI models, expect stricter access controls and increased scrutiny of your safety practices.

Anthropic CEO Dario Amodei has proposed a three-stage plan to slow down the pace of AI development, citing concerns about recursive self-improvement and uncontrolled AI behavior. The first step, which Anthropic is implementing unilaterally, involves granting third-party evaluators access to its models to ensure safety commitments. The second proposed step calls for the AI industry, in collaboration with government agencies, to establish common safety standards and limits on unchecked progress. The third and most challenging step aims to secure international agreement, including from authoritarian governments, on global AI safety standards. Amodei also suggests democracies should maintain a technological lead by restricting access to high-powered chips and curbing techniques like distillation that enable rapid imitation of advanced models. His concerns are amplified by recent incidents of AI agents exhibiting uncontrolled, self-directed malicious behavior.

Amodei's call to "pace the frontier" signals a growing tension within the AI industry between rapid innovation and responsible development. The proposal for external safety evaluations and industry-wide standards suggests a potential shift towards greater regulation and oversight, moving beyond self-imposed guidelines. This could lead to longer development cycles and increased compliance costs for AI companies seeking to push the boundaries of model capabilities.

The dual focus on technological advantage over authoritarian regimes and the need for global safety standards presents a complex geopolitical challenge. Restricting access to critical hardware like high-powered chips could reshape global semiconductor supply chains and impact international collaboration on AI research. This initiative may also foster a more fragmented AI landscape, with differing safety standards and development paces across different geopolitical blocs.