Measuring the sloppiness of code

Code generated by AI assistants is twice as verbose and complex as human-written code.

LLMs have achieved near-perfect code generation, but this code often suffers from "sloppiness," characterized by unnecessary abstractions, duplication, and poor design decisions, leading to massive increases in lines of code (LOC). This sloppiness is difficult for both humans and AI agents to manage, as current evaluation methods for code quality often rely on subjective human intuition rather than objective metrics. The article proposes measuring code sloppiness using metrics like verbosity (duplication and unnecessary verbose lines) and erosion (concentration of mass in large, complex functions). Experiments show LLM-generated code is roughly twice as verbose and eroded as human-written code, with verbosity averaging 0.33 vs. 0.15 and erosion averaging 0.68 vs. 0.31. Iterative coding processes, mimicking real-world agent use, reveal that state-of-the-art models achieve a 0% pass rate when strict evaluation is applied across multiple rounds.

The metrics of verbosity and erosion offer a quantifiable approach to identifying "code sloppiness" that current AI evaluation benchmarks fail to capture. Verbosity, measured by the proportion of duplicated or unnecessary lines relative to total LOC, and erosion, assessing the concentration of complexity in large functions, provide concrete numbers that highlight the inefficiency in LLM-generated code. These metrics reveal that AI code is not just functionally correct but also significantly less efficient and harder to maintain than human-written code, suggesting a fundamental gap in current AI development and deployment practices.

The inability of even state-of-the-art LLMs to maintain a 0% pass rate in iterative coding evaluations underscores the difficulty of managing AI-generated sloppiness in real-world scenarios. This suggests that the current trajectory of simply increasing code generation volume without addressing code quality and maintainability could lead to unmanageable technical debt. Future development must focus on integrating more sophisticated, objective evaluation metrics into the AI development lifecycle to ensure that AI-assisted coding leads to sustainable and efficient software development, rather than an explosion of unmaintainable code.