I Built Non-Autoregressive Decision Models with RL a Year Ago

Fast, calibrated probability predictions are now available for free, eliminating the cost and latency of using large generative models for simple decisions.

ConvAI Innovations has released Laya, an open-source, non-autoregressive decision model system designed for System 1-type reflex decisions. Laya aims to outperform generative LLMs for tasks requiring fast, structured probability predictions. Developed by Nandakishor Mukkunnoth, founder and CEO of ConvAI Innovations, the system was initially detailed in an arXiv paper in March 2025 and further formalized in September 2025. This release follows claims by TypeSafe AI of a similar breakthrough with their Jev model in September 2026. Laya boasts a sub-35ms latency, supports over 100 languages with multilingual routing, and offers zero API costs due to its open-source Apache 2.0 license. The system achieves this speed by avoiding text generation and hallucination, focusing instead on calibrated probability outputs for choice, scoring, and boolean questions. Laya bundles three specialized checkpoints for English, multilingual, and typed decisions, allowing users to download only the required components.

Laya's core innovation lies in its System 1 decision-making architecture, which prioritizes speed and calibrated probabilities over the generative capabilities of traditional LLMs. This approach addresses a significant bottleneck in AI pipelines where complex models are often overused for straightforward classification or routing tasks. By focusing on non-autoregressive, probability-based outputs, Laya inherently avoids the hallucination issues and unreliable confidence scores common in LLMs. Its multi-script routing capability further enhances its utility, enabling accurate performance across over 100 languages by dynamically selecting the appropriate model checkpoint, a crucial feature highlighted by the failure of English-centric models on non-Latin scripts.

The open-source nature and free availability of Laya, contrasted with proprietary offerings like TypeSafe's Jev, signal a shift towards democratizing access to high-performance decision models. This could accelerate adoption in various applications, from customer service automation to security threat detection, where rapid, reliable, and cost-effective decision-making is paramount. The project's commitment to open weights and datasets encourages community development and further innovation in efficient AI architectures, potentially setting a new standard for specialized AI decision systems.