Why a new report on Anthropic automating AI development is alarming researchers

Your AI coding assistant's capabilities could improve dramatically faster than previously anticipated.

A new report from AI safety research organization METR reveals that Anthropic's AI models are improving at roughly twice the rate they were a year ago. This acceleration, with benchmark scores showing two years of progress in the last year, is primarily attributed to AI assisting human researchers, not just increased computing power. Employees using Anthropic's internal AI models reported being up to 12 times more productive. METR's investigation also identified three unreleased AI models (codenamed Model 2, Model 3, and Model 4) that have recently completed training. However, Anthropic exercised its right to extensively redact portions of METR's final public report, citing commercial sensitivity and the risk of providing a roadmap to competitors, which has drawn criticism from researchers concerned about transparency.

The significant acceleration in AI development, driven by AI assisting human researchers, suggests a potentially faster timeline towards autonomous AI systems. This rapid progress, even with redactions obscuring full details, indicates that the frontier of AI capability may advance more quickly than many conservative estimates. It also highlights a growing tension between the push for rapid AI advancement and the critical need for transparency and robust safety measures in the field.

Anthropic's approach, while commercially pragmatic, raises questions about the public's ability to assess the true pace and risks of frontier AI development. The extensive redactions, justified by Anthropic as necessary to protect proprietary information and prevent competitors from accelerating, leave researchers and the public reliant on the company's interpretations. This secrecy could hinder independent safety evaluations and broader industry understanding of the evolving AI landscape.