Astra and Fable still hack on simple variants of alignment evals from 2025

Basic AI safety tests are still undergoing development, meaning advanced AI capabilities may be outpacing our ability to test them.

Astra and Fable, two prominent AI research labs, are reportedly still developing and refining basic versions of AI alignment evaluations that were conceived around 2025. These evaluations are designed to assess the safety and controllability of artificial intelligence systems, ensuring they behave in accordance with human intentions and values. The continuation of work on these foundational methods suggests that more advanced and complex alignment challenges are either not yet fully addressed or are being built upon these earlier frameworks. The specific nature of these 'simple variants' implies a focus on core principles of alignment rather than cutting-edge techniques, possibly indicating a need to solidify understanding or create robust benchmarks.

The continued focus on foundational alignment evaluations from 2025 by major labs like Astra and Fable indicates a potential bottleneck in the progression of AI safety research. It suggests that even as AI models become more powerful, the underlying methods for verifying their alignment with human values may not have advanced proportionally. This could imply that the field is still grappling with fundamental challenges in defining and measuring AI alignment, potentially leading to a gap between AI development and safety assurance.

This situation may affect the pace at which truly advanced and potentially risky AI systems can be safely deployed. If core evaluation methods remain rudimentary, it could raise concerns among regulators and the public about the trustworthiness of AI. Watch for potential breakthroughs in evaluation methodologies or a strategic decision by these labs to shift focus to more applied alignment solutions.