US | Incident Report
Hallucination
2025-12-20
Motion alleges San Francisco judge’s order contained AI-like citation and quote errors
AI Model: Unspecified generative AI tool
I. Executive Summary
A San Francisco Chronicle report described a litigant’s motion alleging that a San Francisco Superior Court judge’s order contained multiple legal citation and quotation problems consistent with AI-style errors. The motion sought reconsideration in the litigant’s case against Elon Musk, asserting that the errors materially affected the decision. The report noted the judge had already issued an amended version of the order once after an error was flagged. The court did not confirm AI use, but the episode underscored rising concern over AI-assisted drafting in judicial workflows.
II. Key Facts
- A party filed a motion for reconsideration asserting that an order contained multiple citation/quotation defects consistent with “AI hallucinations.”
- The motion argued the defects affected a key ruling in the Elon Musk litigation.
- The judge had already issued an amended order once after an error was raised.
- Court representatives declined comment on the specific case, per reporting.
III. Regulatory & Ethical Implications
Highlights governance and accountability gaps when courts experiment with AI-assisted drafting: error correction, transparency, and standards for “substantial” AI contribution. Even absent a finding of AI use, the incident increases litigation risk around record integrity, due process arguments, and potential future policy tightening on judicial AI tools and disclosure.