Twelve years of distributed SQLite

Managed distributed SQLite services are now available to purchase, making it a platform feature rather than a self-hosted deployment.

Twelve years ago, Philip O'Toole launched rqlite, an open-source project that combined SQLite with the Raft consensus protocol to create a distributed relational database. This innovation allowed data to be replicated across a cluster, ensuring durability even if a node failed. Initially, rqlite was the sole project in the "distributed SQLite" category. By 2017, Canonical developed dqlite, a similar embedded library for its products. In 2021, Litestream emerged, offering a different approach focused on disaster recovery for single-node SQLite by streaming write-ahead logs to object storage, arguing that most applications don't need a full cluster. This sparked a broader "SQLite is enough" movement. Since then, the landscape has expanded with projects like cr-sqlite introducing CRDTs for multi-writer divergence, libSQL forking SQLite for custom engine changes, and platforms like Cloudflare (D1) and Turso offering managed, edge-located SQLite services, marking a shift from self-hosted solutions to commercially available platforms.

The evolution of distributed SQLite highlights a clear design space with three distinct approaches: CP (consensus-based for strong consistency, like rqlite), disaster recovery (streaming for cheap recovery, like Litestream), and AP (conflict-free replicated data types for eventual consistency, like cr-sqlite). These different strategies cater to varied application needs, from high availability to local-first synchronization. The commercialization of these ideas, with venture funding and acquisitions for platform offerings, indicates a maturing market seeking to monetize infrastructure primitives.

The shift from open-source projects to managed platforms signals a significant change in how developers can access and utilize distributed SQLite capabilities. While foundational open-source projects like rqlite remain available for self-hosting and deep understanding, the increasing availability of these solutions as a service from major cloud providers and specialized startups suggests a future where ease of use and managed infrastructure become the primary drivers for adoption.