METR: Anthropic redacted much of the "most informative" evidence from METR's 100-page report, so readers "must rely on our judgment more than we consider ideal"

The primary AI safety research organizations can no longer independently verify claims made by AI developers about their risk assessments.

METR, a nonprofit focused on AI safety, has released a report evaluating Anthropic's internal assessment of AI development acceleration risks. METR gained access to a nonpublic version of Anthropic's August 2026 Risk Report, specifically reviewing sections on "Risks from automated R&D" and capability thresholds. METR's assessment team, which included two employees who previously embedded with OpenAI, focused on whether AI itself is speeding up AI development and if Anthropic had reached critical AI capability milestones. The report states that Anthropic redacted "most informative" evidence, forcing METR to rely more on their own judgment than ideal. METR also reviewed Anthropic's assessment of "Misalignment in high-stakes settings," agreeing that catastrophic harm risk from misalignment in covered models is low but noting significant disagreements with Anthropic's analysis.

The redacted evidence means that the public cannot independently assess the validity of Anthropic's claims regarding AI acceleration risks. This lack of transparency complicates efforts to establish universal safety standards or regulations for advanced AI development. It also suggests a potential pattern where AI developers control the narrative around their own safety measures, limiting external scrutiny.

METR's public statement about the redactions indicates a growing tension between AI developers seeking to demonstrate safety and independent evaluators demanding full transparency. This situation may lead to increased calls for mandatory third-party audits and more stringent data-sharing requirements for AI safety reports, particularly as AI capabilities advance.