AI Hallucinations Are in Court Filings Again. Lawyers Keep Acting Surprised.

A Wall Street law firm filed documents containing AI-generated hallucinations, and r/law received the news the way a school nurse receives word of another kid eating glue: resignation, a little dark humor, and the unspoken knowledge that it will happen again.[¹] The post linking to the story drew almost no debate — because there was nothing left to debate. The citations were fake, the attorneys were embarrassed, and the pattern has repeated enough times that the r/law community has developed something like a filing-incident fatigue. What's more telling than the incident itself is that, in the same week, someone shared a podcast on how using Claude changes attorney-client privilege, calling it worthy of continuing legal education credit.[²] The two posts appeared days apart, pointing in opposite directions: one documenting the failure mode, one trying to build the competency. The profession is having both conversations simultaneously and neither loudly enough.

This is the particular quality of the AI and law moment right now — not crisis, not integration, but a slow institutional reckoning with a technology the profession adopted faster than it could govern. Lawyers have been sanctioned, publicly, more than once for AI-hallucinated citations. The Pennsylvania sanction that made headlines a few weeks ago wasn't an aberration; it was a precedent. And yet the conversations in legal communities still treat each new incident as a fresh surprise rather than a predictable output of a system where incentives reward speed over verification. A solo practitioner using Claude to draft a brief because billing hours are tight is making a rational choice. The hallucination risk is abstract right before the filing deadline; the saved time is concrete.

The deepfake problem is where the stakes get harder to dismiss. A news report circulating this week documented that AI deepfakes are poised to enter court proceedings — not as a future concern but as a present one — at a moment when trust in the legal system is already in poor shape.[³] The implications run in two directions at once: deepfakes as tools to fabricate evidence, and deepfakes as an alibi for authentic evidence being declared fabricated. Both are genuinely corrosive, and both are already happening at the margins. The legal community doesn't yet have a reliable framework for authenticating digital evidence in a world where any audio or video can plausibly be contested. Across the broader AI and misinformation conversation, the deepfake problem has been treated as a media and politics issue; courts are where it becomes a due process issue, which is a different order of problem entirely.

What's missing from the legal AI conversation — and conspicuously absent from r/law this week — is anything like a structural response. The posts are individual: one hallucination incident, one podcast recommendation, one question about whether tariff-era price hikes are actionable. The Hangzhou court's public hearing on AI agent traffic hijacking as an unfair competition case represents the kind of doctrinal development that will eventually force U.S. courts to articulate their own positions, but that story is barely circulating in English-language legal communities.[⁴] Meanwhile, the AI regulation conversation keeps producing frameworks — Deloitte on model validation, Lawfare on Grok and accountability — without producing enforcement. The law is the one institution theoretically equipped to make AI accountability real, and it's still mostly processing AI as a novelty rather than a permanent feature of the evidentiary and contractual landscape it governs.

The honest read on this week is that the legal profession is roughly eighteen months behind where it needs to be, and the gap isn't narrowing. Firms are adopting AI tools faster than bar associations are producing guidance, faster than courts are updating evidentiary rules, and faster than any individual attorney can track the liability implications of a tool that confidently invents citations. When AI liability is the question nobody wants to answer, it tends to fall to courts to answer it by default — through sanctions, through rulings, through the slow accumulation of case law. That process has started. It's just moving at the speed of litigation, which is to say: much slower than the technology.