Anthropic says it is giving employee-like access to METR as part of its pledge to partner with evaluators for assessment of its models and safety practices

Third-party AI model evaluations are now published without veto power, increasing scrutiny and establishing new benchmarks for AI safety.

Anthropic has begun granting extensive, "employee-like" access to its AI models and safety practices to METR, a nonprofit organization focused on evaluating advanced AI. This move aligns with Anthropic's commitment to third-party safety assessments and its signing of the White House Accord on Super Intelligence, which mandates partnerships with independent auditors. METR will receive company hardware, office space, and access to internal tools comparable to Anthropic's own risk assessment teams. While certain sensitive information like customer data and proprietary model architectures will be restricted, METR will have broad visibility into Anthropic's operations and will publish its findings. Anthropic states it cannot veto METR's publications but can request redactions for narrowly defined, pre-agreed categories, with METR able to comment on the redactions. The company is also establishing tiered access for other evaluators like SecureBio and Accenture, with the goal of increasing transparency and accountability in AI safety.

Anthropic's approach of providing "employee-like" access to METR signals a significant evolution in AI safety evaluation, moving beyond audits to embedded, continuous oversight. This deep integration aims to bridge the knowledge gap between AI developers and independent assessors, mirroring models seen in financial regulation. By allowing a nonprofit unfettered publication rights, Anthropic is setting a precedent for transparency that could pressure other leading AI labs to adopt similar rigorous, external validation processes for their models and safety protocols.

This initiative creates a more robust ecosystem for AI risk assessment, where the publication of findings, even with limited redactions, provides invaluable data for researchers and the public. The tiered access model suggests a scalable framework for independent evaluation that can be adapted as AI capabilities advance. Companies like Anthropic are proactively addressing concerns about AI's rapid development by inviting external scrutiny, which may become an industry standard for responsible AI deployment and governance.