Dessa: the most authentic full-duplex speech model ever built

Voice AI can now understand and generate speech with human-like emotion and timing, improving interactions.

Block, Inc. has unveiled Dessa, a novel full-duplex speech model trained exclusively on human conversation audio, bypassing text entirely. Developed by a small AI research lab within Block, Dessa learns nuances like emotion, sarcasm, and conversational timing directly from sound. Unlike traditional models that convert speech to text, Dessa operates as a true speech-to-speech system, processing audio input and generating audio output in real-time. The model can infer emotional states, respond with appropriate vocal inflections, and even interject naturally into conversations. Dessa was trained on a single NVIDIA GB200 NVL72 and can run locally on high-end MacBooks. The lab is extending early access to AI safety researchers to identify potential risks.

Dessa's speech-only training approach represents a significant departure from conventional LLMs, which rely heavily on vast text datasets. This method allows Dessa to capture the full spectrum of vocal communication, including non-verbal cues like laughter and sighs, which are typically lost in text-based conversions. The model's ability to perform true speech-to-speech processing in real-time, with full-duplex capabilities, suggests a future where AI interfaces are more intuitive and emotionally resonant. This could profoundly impact applications ranging from customer service bots to personal assistants, making them feel less robotic and more genuinely interactive.

The implications for AI safety and human-computer interaction are substantial. By learning directly from the complexities of human speech, Dessa may offer new insights into how AI can better understand and respond to user emotions, potentially preventing misunderstandings or dangerous situations where current AI remains oblivious. The lab's proactive outreach to safety researchers indicates an awareness of the novel risks associated with such an advanced speech model. As Dessa becomes more accessible, it will likely spur further research into audio-centric AI development and the ethical considerations surrounding emotionally aware artificial intelligence.