Progress on provable inference for large language models

You can verify that an AI model's responses are from the specific model advertised, not a substitute. A new system called FlashProve makes this possible by generating cryptographic proofs that are fast enough for practical use.

Anthropic has developed a system called FlashProve that uses zkSNARKs to provide cryptographic proofs of AI model inference. This allows users to verify that responses originate from a claimed model, addressing concerns about model substitution or tampering by malicious actors or routing services. The system hides model weights but not the model architecture. FlashProve offers significant speed improvements over prior open-source systems, generating proofs at approximately 110 tokens per second for a 400B-parameter model on a single H100 GPU, with proof sizes around 19MB per 1,024-token request. Anthropic has released the system as open-source, including executable code, formal verification proofs in Lean, and technical documentation, with the goal of accelerating research and demonstrating the practicality of zkSNARKs for provable inference. This progress has implications for AI security, development pacing, customer integrity guarantees, and auditing for regulators.

Anthropic's FlashProve system represents a significant step towards making provable inference a practical reality for large language models. The reported speed of over 100 tokens per second for generating proofs on a 400B-parameter model, while computationally intensive, is orders of magnitude faster than previous approaches. This leap in performance suggests that cryptographic verification of AI outputs may soon move beyond theoretical discussions and become a viable feature for both consumers and enterprise applications, enabling stronger trust and security guarantees.

The implications of FlashProve extend to critical areas like AI development pacing and regulatory compliance. By enabling regulators and auditors to verify that specific models are being used for inference without accessing proprietary model weights, it could streamline oversight processes. Furthermore, its open-source release fosters community development, potentially accelerating the adoption of verifiable AI and providing a foundation for new security paradigms in the evolving AI landscape.