Neki – Sharded Postgres

You can now scale PostgreSQL databases horizontally without losing compatibility or extensions.

PlanetScale has launched Neki, a sharded PostgreSQL database now available in platform preview. Neki allows users to scale PostgreSQL across multiple machines while maintaining standard PostgreSQL functionality on each shard. Applications connect to Neki routers, which use the standard PostgreSQL wire protocol, ensuring compatibility with existing drivers and ORMs. Each shard consists of a full PostgreSQL cluster with a primary and replicas across availability zones, without custom storage engines. Users define how data is distributed via a JSON data topology. Neki offers built-in, fully online workflows for operations like schema changes, upgrades, failovers, and resharding. It also includes existing PlanetScale features such as Insights, schema recommendations, and branching. Neki can initially be run unsharded and later resharded as a seamless workflow.

Neki addresses a significant gap in the cloud PostgreSQL market by offering a natively sharded solution that maintains full PostgreSQL compatibility. Unlike other distributed SQL databases that abstract away shard keys or use proprietary engines, Neki keeps real PostgreSQL on each shard, allowing for the use of extensions and familiar SQL behavior. This approach aims to alleviate common scaling bottlenecks faced by growing PostgreSQL applications, such as vacuuming, indexing, and backup times, by distributing the load across multiple machines. The platform preview suggests an iterative development approach, with PlanetScale likely incorporating feedback to refine the product before a general availability launch.

The introduction of Neki signals a broader trend towards making complex database scaling patterns more accessible to developers. By abstracting the intricacies of sharding through online workflows and a clear data topology configuration, PlanetScale is lowering the barrier to entry for horizontal database scaling. This could impact how businesses architect their data layers, potentially reducing the need for complex application-level sharding logic or migration to less familiar distributed database systems. Companies relying heavily on PostgreSQL for large-scale applications will likely evaluate Neki as a competitive alternative to existing scaling strategies or other distributed database solutions.