Anthropic outlines metrics to track AI development at frontier labs: how much AI R&D is done by AI, how well agents are overseen, and how compute is allocated

You can now find out if AI is building itself faster than humans can build AI.

Anthropic has introduced a new framework to track the development pace of advanced AI systems within frontier labs. The company is proposing three key metrics: the extent to which AI is used to conduct AI R&D, the effectiveness of oversight for AI agents, and the allocation of computational resources. Anthropic is sharing internal data for these metrics, showing that as of August 2026, AI "leads" 26% of its AI R&D work, with over 90% involving AI at an "AI collaborates" level or higher. The company also reported that 100% of its approximately 30,000 research and engineering agents' actions are monitored, with a blocking rate of 0.002% for real-time online monitors and about 1 in 1,000 transcripts flagged for offline review. Anthropic plans to embed third-party evaluators to verify these safety practices and metrics, aiming to provide greater public visibility into AI development.

Anthropic's initiative signals a growing industry recognition of the need for transparency in AI development, moving beyond just capability evaluations to encompass the R&D process itself. By quantifying AI's role in its own advancement and the oversight mechanisms in place, companies can provide a more nuanced view of progress, which is crucial as AI systems become more autonomous and integrated into research pipelines. This framework could set a precedent for how other frontier labs report on their internal development, potentially influencing safety standards and regulatory discussions.

The proposed metrics, particularly the AI-led R&D and agent oversight measures, offer a way to gauge the speed of potential recursive self-improvement and the robustness of safety controls. The data shared by Anthropic, while internal, provides a benchmark for what is currently measurable and auditable, highlighting the challenges and potential solutions for external verification. As AI agents become more prevalent in complex tasks, the metrics for monitoring their actions and the effectiveness of oversight will become increasingly critical for ensuring responsible development and deployment.