Anthropic gives outside AI watchdog “employee-like” access

You can now ask the AI models you use for more accurate safety assurances because their developers have committed to independent, unprecedented levels of external scrutiny.

Anthropic has granted the nonprofit AI watchdog METR what it calls "employee-like access" to its internal systems and hardware to independently assess the safety risks of its AI models. This move follows through on a commitment by AI labs to enhance safeguards as AI capabilities advance. METR, which has previously worked with other major AI developers, will embed a team within Anthropic's offices. The evaluators will have access to internal tools and systems comparable to Anthropic's own risk assessment teams. Anthropic stated that this transparency is necessary because while some safety information cannot be shared publicly due to risks of misuse, accountability is paramount. This arrangement allows METR to publish its findings without Anthropic's prior approval, though Anthropic may request redactions for sensitive data.

The integration of METR within Anthropic signifies a substantial step towards industry-wide self-regulation and a potential model for how AI safety will be managed as models become more powerful. This approach aims to preempt stricter government oversight by demonstrating a commitment to rigorous, independent risk assessment, akin to bank supervision, before potential catastrophic failures occur. The ability for METR to publish findings without veto, while allowing for redactions of specific sensitive data, offers a balanced approach to transparency and security.

This development is particularly crucial for Anthropic as it navigates a potential $2 trillion valuation IPO, where investor confidence in safety protocols is paramount. The voluntary nature of this embedment suggests an industry-wide trend toward proactive safety measures, with METR positioned as a key independent validator. The funding model for METR, drawing from individual and philanthropic sources rather than AI companies, aims to mitigate conflicts of interest and bolster the credibility of its assessments.