Block unveils Dessa, a "true speech-to-speech" AI model trained on "nothing but human conversation" that it says runs locally on a MacBook Pro M4 Max or better

Your voice assistant will understand tone and emotion, and respond more naturally.

Block, Jack Dorsey's financial technology company, has launched Dessa, a new artificial intelligence model designed for "true speech-to-speech" interaction. Unlike many current AI systems, Dessa is trained exclusively on human conversation, meaning it learns language and nuances directly from spoken words without relying on written text. This approach aims to create a more natural and intuitive voice interface. Block demonstrated Dessa's capabilities through sample conversations, including a voice-based guessing game and discussions about films, highlighting its ability to maintain a natural conversational flow. The company claims Dessa is capable of running locally on high-end consumer hardware, specifically mentioning a MacBook Pro M4 Max or superior. Block stated its motivations for developing Dessa include curiosity, improving human-AI understanding, and enhancing AI safety by enabling models to better perceive and respond to emotional cues in human speech. Early access to Dessa will be provided to a select group of AI safety researchers.

Dessa's "true speech-to-speech" architecture, trained solely on conversational audio, represents a significant departure from text-centric AI development. This method aims to capture the subtleties of human speech, including timing and emotional inflection, which are often lost in translation to text and back. By processing and generating audio directly, Dessa promises a more fluid and empathetic interaction, addressing key limitations in current voice AI that flattens human expression. The ability for such a model to run locally on consumer hardware also suggests a future where advanced AI communication tools are more accessible and private.

The emphasis on "safety" by training on human conversation and processing vocal emotion signals a push towards more human-aligned AI. Block's approach directly tackles the risk of AI lacking emotional awareness, which could lead to problematic interactions in sensitive situations. By prioritizing an AI that can 'hear' and react to distress or other emotions, Block is positioning Dessa as a foundational step for future AI systems that are not only intelligent but also more considerate. This focus on inherent safety through design may influence how future voice-based AI agents are developed and regulated.