Lawyer fined $5K over AI-hallucinated witnesses in a murder case

Legal briefs now carry a risk of AI-generated falsehoods that can result in financial penalties.

New Mexico lawyer Stephen Aarons has been fined $5,000 and held in contempt by the state's Supreme Court for submitting a legal brief that included AI-generated, fabricated witnesses and fake police testimony. The court's filing on Wednesday stated the brief contained false testimony from nonexistent witnesses and inaccurate details about a shooter's appearance. Aarons admitted to using ChatGPT for a summary of a murder case, believing it would be accurate, but Justice C. Shannon Bacon questioned his awareness of AI's known tendency to 'hallucinate' or fabricate information, noting that the issue is frequently reported in the news. This case follows similar instances where legal professionals have been penalized for relying on AI-generated content without proper verification, including fines for submitting briefs with false legal citations and quotations.

The significant fine and contempt charge against Stephen Aarons underscore the judicial system's increasing intolerance for unverified AI-generated content in legal filings. As AI tools become more accessible, their propensity for factual inaccuracies, commonly referred to as 'hallucinations,' poses a direct threat to the integrity of legal arguments and evidence. This ruling serves as a stern warning to legal professionals that the responsibility for verifying all submitted information, regardless of its source, remains solely with the attorney, highlighting a critical compliance challenge for the legal tech sector.

The incident reflects a broader industry-wide concern about the responsible deployment and use of generative AI in professional contexts where accuracy is paramount. For legal practitioners, this means integrating AI tools with robust human oversight and verification processes, rather than treating them as autonomous sources of truth. The court's emphasis on Aarons's alleged ignorance of AI risks suggests a potential future where legal education and bar association guidelines may need to explicitly address AI literacy and ethical usage to mitigate similar recurrences.