In recent years, we have seen numerous cases where attorneys used generative AI and were caught including fake citations, quotes, and other major errors in their filings. This typically plays out in court dockets, where opponents or judges spot the fabrications and scold them for wasting everyone's time. While these blunders sometimes result in serious sanctions, a recent appeal hearing captured this exact phenomenon unfolding live on camera, with an attorney caught red-handed using what are highly likely AI-fabricated citations.
On May 20, in the Supreme Court of the State of New York Appellate Division, Justices Valerie Brathwaite Nelson and Hector LaSalle reamed out lawyer Michael Sanders for more than 20 minutes, calling the situation "striking, concerning, disappointing, and saddening." The plaintiff, Judith Landberg, is suing the city of New York after tripping on sidewalk bricks. Sanders, representing the plaintiff, was attempting to argue the definition of a sidewalk when the confrontation began.
"In preparing for this oral argument and reviewing the brief of appellant, it came to the attention of the court that the brief submitted by plaintiffs cites at least three cases that appeared to be fictitious," Justice Nelson said. "None of these cases, nor the quoted language, appears to exist." Furthermore, Nelson noted that Sanders cited 10 other cases that misrepresented the law. When asked to respond, Sanders stumbled, stating he was unprepared to speak on those specific citations.
Nelson promptly cut him off, citing Rule 3.3 A of the rules of professional conduct, which strictly prohibits lawyers from knowingly making false statements of fact or law to a tribunal. Sanders stammered and apologized, claiming he did not know where the citations came from. While the judges did not explicitly mention large language models (LLMs), this incident fits the exact profile of AI-generated legal hallucinations that have plagued courts nationwide.
[AgentUpdate Depth Analysis] This case highlights the persistent vulnerability of LLM hallucinations in high-stakes professional fields like law. While general-purpose AI chat tools offer rapid drafting, they lack the rigorous verification layers required for professional workflows. To transition from simple text generators to reliable AI Agents, the industry must move beyond basic prompting towards advanced RAG (Retrieval-Augmented Generation) and multi-agent systems. A specialized legal Agent must integrate an "Editor Agent" or "Verification Agent" that performs cross-reference checks against official databases (such as Westlaw) before final output. For the broader AI Agent ecosystem, this case is a stark reminder that "autonomous execution" without strict deterministic checks is a liability; true enterprise-grade Agents must balance LLM creativity with strict verification protocols to bridge the trust gap.
