Dessa: Conversation Model trained only on speech

Voice interactions with AI gain emotional depth and natural timing, moving beyond stilted text-based exchanges.

Block, Inc. has introduced Dessa, a novel conversational AI model trained exclusively on human speech rather than text. Developed by a small AI research lab within Block, Dessa is designed to mimic natural human conversation, incorporating elements like emotion, nuance, pauses, and cadence. Unlike traditional AI models that convert speech to text, Dessa operates as a full-duplex speech-to-speech system, processing and generating audio in real-time. The model learned language, emotion, and conversational cues directly from audio data, requiring significantly less data than text-based LLMs. Dessa can also infer and express emotions, sarcasm, and humor, and it can engage in real-time conversation with other AI models. The lab is also seeking AI safety researchers to assess Dessa's potential risks.

Dessa represents a significant departure from conventional AI development by prioritizing auditory learning. This approach could lead to AI that is more intuitive and empathetic, bridging the gap between human and machine communication by capturing the richness lost in text conversion. The model's ability to process speech in real-time and incorporate non-verbal cues suggests a future where AI assistants are more integrated into natural human interactions.

This development signals a potential paradigm shift in how AI understands and processes language, moving towards more holistic comprehension. For AI safety researchers, Dessa presents a unique opportunity to explore risks inherent in models that learn from raw human expression. The efficiency gains in training data also suggest a more accessible path to developing sophisticated AI systems, potentially democratizing advanced AI capabilities.