Whistle: Speech to Text in 16.9 MB

The cost to run speech-to-text locally on devices is now 16.9 MB, a fraction of previous requirements.

Cactus Compute has released Whistle, a new open-source speech-to-text model designed for on-device processing. The model is notably small, at just 16.9 MB, and runs on the CPU without external dependencies. Whistle supports transcription in seven languages, achieving a first token in 11 milliseconds. It integrates with Cactus Compute's existing Needle model, allowing a single binary to process audio directly into tool calls. The model provides transcription, word timestamps, and speech embeddings, all processed locally on the device. Benchmarks show Whistle outperforming Whisper and Moonshine on several metrics, including word error rate on specific datasets and time-to-first-token, despite being significantly smaller than Whisper.

Whistle's 16.9 MB footprint signifies a major advancement in on-device AI, particularly for edge computing and privacy-sensitive applications. By eliminating the need for cloud-based processing, it reduces latency and bandwidth requirements, making sophisticated speech recognition feasible for wearables, IoT devices, and embedded systems previously constrained by size and power limitations. This move democratizes access to real-time speech processing, enabling new categories of intelligent, always-available interactions without data leaving the user's device.

The integration with Needle, allowing direct conversion of speech to tool calls within a single binary, signals a shift towards more cohesive agentic systems. Developers can now build applications that seamlessly incorporate voice interfaces without the complexity of managing separate transcription services and command parsing logic. This architecture simplifies deployment and opens up possibilities for more responsive and context-aware user experiences across a wide range of hardware, from smartphones to microcontrollers.