We should be more ambitious with the software we build

Building highly optimized or novel software components is now achievable in a fraction of the time previously required.

Michael Malis, known for the pgrust project, is advocating for more ambitious software development, asserting that current capabilities allow for tackling projects previously deemed infeasible. His project, pgrust, a performance-focused rewrite of parts of Postgres in Rust, has uncovered significant bugs within Postgres itself, including numerical, cache, pgcrypto, and memory safety issues. While pgrust aims for performance gains, the bugs found and fixed represent a more direct contribution to improving the existing software. The author notes skepticism towards rewrite projects due to the prevalence of low-quality, AI-generated alternatives, but highlights Malis's thorough testing as evidence of pgrust's serious intent. Even if pgrust doesn't succeed, the optimizations explored could influence other database systems like Clickhouse.

The advancements in AI and development tools are significantly lowering the barrier to entry for complex software projects. This means individuals and small teams can now tackle ambitious initiatives, such as performance-critical rewrites or specialized tooling, that were once only feasible for large, well-funded organizations. The author's personal experience building faster regex engines, optimizing Rust compilation, and even creating superhuman board game AI in mere hours exemplifies this shift, indicating a future where more specialized and performant software can emerge from unexpected places.

This trend suggests a potential democratization of high-performance software development, enabling creators to focus on ambitious projects without being overly constrained by the limitations of existing tools or the sheer time investment required. While AI-generated code often results in low-quality output, sophisticated models, when guided by knowledgeable developers, can accelerate development cycles dramatically. This capability allows builders to push the boundaries of what's possible, leading to the creation of more sophisticated and performant software across various domains.