RIP, vector database

Vector search is no longer the only primary index for a major search database.

Turbopuffer is overhauling its storage architecture with the upcoming release of turbopuffer v3, informally dubbed "tpuf v3". This new engine will fundamentally alter how documents and indexes are organized, written, compacted, and queried, aiming to support significantly more query plans at a much larger scale. Initially launched as a serverless vector database optimized for cheap and fast vector searches, utilizing object storage and tiered NVMe SSD/memory caches, Turbopuffer has evolved into a more generalized search database. The existing architecture, with the ANN vector index as its primary index, has been pushed to its limits. Turbopuffer v3 will relegate ANN to a secondary index, introducing a new primary index to address limitations in storage amplification, write amplification, and vectorization that have hindered performance for non-vector query types. This foundational change follows extensive development, with the company now focusing on performance tuning and benchmarking.

The shift from an ANN-primary index to a generalized primary index signifies a maturing market where specialized vector databases are broadening their capabilities to compete with more comprehensive search solutions. This evolution caters to applications requiring a blend of vector search alongside traditional database operations like attribute filtering and full-text search, suggesting that hybrid search architectures are becoming the standard. Companies like Turbopuffer are thus moving beyond niche applications, aiming to become all-in-one solutions for complex data retrieval needs.

This architectural pivot by Turbopuffer indicates a trend towards database systems that can efficiently handle diverse query types without compromising performance. The focus on overcoming storage and write amplification issues, and enabling better vectorization for non-ANN queries, points to a future where databases are optimized for compute efficiency and flexibility. Developers can expect more integrated search experiences that offer competitive performance across a wider range of search paradigms, potentially reducing the need for multiple specialized databases.