If coding is solved, what now?: Measuring the sloppiness of code

The complexity and cost of maintaining AI-generated code could soon outweigh its benefits.

LLM-generated code, while often formally correct, can be "sloppy," introducing unnecessary abstractions, duplication, and poor design decisions. This "slop" makes code difficult for humans to manage as projects rapidly expand in size. Current methods for measuring code sloppiness are inadequate; LLMs themselves are poor judges of their own code, and human evaluation is not scalable. New metrics like verbosity, which measures duplicated and verbose lines relative to total lines of code, and erosion, which quantifies the concentration of complexity in large functions, show promise. Comparative analysis reveals that LLM-generated code is approximately twice as verbose and eroded as human-written code. Furthermore, state-of-the-art models struggle with iterative coding tasks that mimic real-world development, achieving a 0% pass rate in some evaluations, highlighting the accumulation of bad decisions over time.

The emerging challenge in AI-driven software development lies not in generating functional code, but in managing its quality and maintainability at scale. Metrics like verbosity and erosion offer a quantitative approach to assessing "code slop," distinguishing between LLM-generated output and human-written code. This distinction is critical as the industry grapples with the long-term implications of rapidly increasing codebases that may become unmanageable due to inherent inefficiencies.

The findings suggest a potential bottleneck for autonomous AI agents: their inability to inherently produce clean, maintainable code without human oversight or significantly improved evaluation frameworks. As LLMs become more integrated into development pipelines, the focus must shift from mere code generation to robust quality assurance that accounts for architectural soundness and long-term project health, rather than solely relying on test-case passing.