1/67
@burrytracker
And it begins
Sullivan & Cromwell just admitted to a federal judge its court filings contained AI hallucinations
The firm apologized to the federal judge as they had to submit multiple corrections focused around:
• Fictitious Case Names: The filing included names of legal cases that do not exist
• Fabricated Quotes: The document contained direct quotes that were never actually spoken or written
• Non-existent Statutes: The AI incorrectly analyzed or entirely invented provisions within the U.S. Bankruptcy Code
The primary team and secondary review all failed to catch these errors, meanwhile the firm's partners bill $2,000+ per hour
2/67
@TauchTrader
Curious what model they were using.
3/67
@burrytracker
The articles don’t specify, however, the firm advises OpenAI
4/67
@elonmusk
Hmm
5/67
@TinfoilTricorn
sounds like an intentional fail rather than an accidental one, all it would take is running it past 3 different models generally to find the fake errors. ChatGPT 5.4 extended, Claude Opus 4.6 extended thinking, Grok 4.20, I don't think they really did an external validation or multi model verification. Fake errors like this is 2024 level work.
6/67
@viperr
Charging big law partner rates for AI slop hallucination case study.
Should be an ethics strike lol.
7/67
@The_Analyst
Wonder if they used Harvey or one of the other legal-tech AI unicorns…
8/67
@LastCoinStandng
The article title should be revised; it's no longer a "top law firm".
9/67
@jonbrooks
yeah attorneys are one of the biggest scams. hopefully AI gets good enough to replace 90% of the work that attorneys do.
10/67
@ChristieSmythe
With so much sh*t flying around, it's bound to hit the fan at some point. Just a matter of when.
11/67
@zhaohan_dong
@grok Has the lowest hallucination rate among all the providers rn
12/67
@Kerrpeye
Jeez... I'm a 62 yr old nobody. BUT, I have been in the investigative field for 38 yrs. I conducted my own tests of AI & determined it's completely unreliable. I found it made up legal cases with names & numbers that never existed. Obviously they didn't conduct tests before use.
13/67
@MartinSzerment
Wild that a $2B firm didn’t bother to check citations before filing. AI isn’t the problem—complacency is.
14/67
@ClarkSims12
My ex-wife's lawyer did this the old fashioned way, just making stuff up in court, and citing cases that say the opposite of what he alleged they said in his pleadings. AI just automates the "liar liar pants on fire" strategy of lawfare.
15/67
@MDJD2
I utilize AI regularly in legal work. AI is excellent when you identify the arguments & then give explicit instructions in helping you draft. And if you're overseeing it through the entire drafting the hallucinations happen but you notice quickly & correct. These lawyers are lazy
16/67
@another_heifer
This has been happening for well over a year... multiple law firms & attys caught filing briefs w AI-hallucinated case cites. Attys are supposed to do continuing legal education...
Even so, it's impossible for an atty not to know AI hallucinates case cites as this has happened to others at least 10X b4... and reported in legal newsletters by ABA, state bar orgs, etc.
Severe sanctions and fines must be levied against attys and law firms that submit briefs w fake case cites whether or not one used AI as the atty signing off on the brief is averring that everything contained in the brief is truthful & case references are accurately cited.
Thus, attys are and always had been responsible for checking case cites in the briefs they submit to a court whether or not AI was used.
17/67
@AngelliAngelo
I have had an associate I work with send me 15 separate analyses of legal issues and it has been wrong 100% of the time. Even about something as simple as a filing deadline.
18/67
@InformedMama1
Garbage in. Garbage out.
AI elevates sources with known erroneous & fraudulent data, from Wikipedia & Reddit, to legacy media and of course government websites & scientific studies.
Anyone remember the Covid era?
This is the slop AI is built upon.
19/67
@QuietChart
Beyond concerning! It's a systemic red flag. When partners are billing $2,000+ an hour, the expectation is human expertise, not unchecked AI output.
20/67
@redgreenbl35768
If this is how they reviewed AI outputs, imagine how little review went into junior team member outputs.
21/67
@FrancesJaneB
Stay away from ChatGPT. It makes things up. You have to ask every time if it’s lying.
22/67
@SteveTheobald70
How the hell does a $1500/hour law firm like that screw up so badly?
23/67
@sriramkiron
for the people wondering what model they were using:
ALL the models do this. EVERY SINGLE MODEL hallucinates fake cases/papers/quotes when you ask it to research a topic.
you CANNOT rely on an LLM's output, even if it's sota. fact-check everything, or you will get in trouble.
24/67
@ryanofmaryland
All lawyers involved to be disbarred; poorly represented clients ought to sue them to kingdom come.
25/67
@PBaron9TX
If AI is untrustworthy in law, is it equally untrustworthy in medicine, or engineering or everything?
26/67
@MsVeteranTeach
In other words they didn’t bother to look up the sources they themselves were citing. Anyone involved should be fired but one hold my breath
27/67
@Matt0733
The partners are billing $2,000 an hour while interns crank away on ChatGPT and Claude in the back room
. Shakespeare had the right idea.
28/67
@gabriel_horwitz
Insane story agreed
[Quoted tweet]
Sullivan & Cromwell, whose partners billed several hundred million in the FTX bankruptcy, just apologized to a federal judge for AI hallucinations in a high-profile filing. worth reading what actually happened.
- the filing misquoted the US bankruptcy code. not a foreign statute. the US bankruptcy code. in a bankruptcy filing
- multiple case citations were wrong. one cited case "is not a case." it pointed to a different decision in a different circuit
- opposing counsel found it. not any of the five senior partners assigned to the case
- the firm's own policy instructs lawyers to "trust nothing and verify everything." submitting the filing was a policy violation
- they noted in the same letter that other errors in the matter were "made by humans, not AI." presumably for balance
Imagine paying $2,000/hr for an enterprise ChatGPT license.
29/67
@4854Capital
This isn't an AI problem.
It’s a governance problem.
At those rates, you aren't paying for the prompt.
You are paying for the proofreading. If the humans in the loop aren't actually looping.
The "gold standard" of legal counsel is just another hallucination.
30/67
@thebigjakaboski
As an ex-big firm lawyer, I can say with 99% confidence how this happened. The overwhelming economic incentive at big firms is overbilling. Associates don't meet hours reqs by submitting honest timesheets. Partners don't meet revenue reqs by calling out associates who don't.
31/67
@VelliMach
Made up citations are a huge problem for AI. It's bizarre how this happens systematically. It's like it wants to please you so much that invents citations to be helpful in your case.
32/67
@jamesaholt
Welp. Think the AI budgets at large firms (law and other businesses with a license) are gonna get whacked. I engage many such firms and we all discuss the fact that humans must in the game deciding.
33/67
@jondpratt
The Age of Abundant Fiction...and it's All FREE!
34/67
@jcarole
Just to be clear. AI could be useful. Ask it where are the holes in my logic? How can order this more clearly. It is not, however, able to be accurate at a scholarly level. You must fact check and edit the shyt it does.
Lazy stupid b*stards.
35/67
@ReciprocitEye
So nobody even glanced over that filing before it went out?
That's beyond lazy and crooked.
36/67
@JasonSacks
Oh that is rubbish according to Anthropic, because they can replace all legal software. lol!!!
37/67
@TuiTurd
If using AI the firm should disclose this to their client and confirm how much of the advice/drafting was AI. Then lower their attorney chargeout rates/billable times significantly.
38/67
@TheraPantis
This happens when you ask for something and want it to work in your favor. Doesn’t always mean the evidence supports your framing.
Maybe we could say that hallucination happens when there is not enough evidence to frame something how we would like to.
And I think that framing, in a legal sense, is worth taking pause to consider.
39/67
@JSkar76
Sounds like the graduates of declining law schools have been hired by formerly respected big firms. Can’t imagine even paralegals not being able to spot fictitious case names.
40/67
@DisSpace4Sale
Fair warning...
ai works great for creating pictures, videos, animation, & searching info
BUT ai has the ability to cause Major problems
one ai was caught telling children how to suicide
another ai was proven to be politically biased
there are talks of ai needs watermarks
41/67
@Jen_Esquire
No, other dumb lawyers who should be disbarred are doing this also. Boies Schiller is another “top” law firm that did this.
42/67
@eric_m_freeman
Sounds like Sullivan Cromwell is using Claude and is learning the heartburn of Opus 4.6/4.7
43/67
@ChiyoSolana
It's a huge issue that firms think they can outsource real work to AI and the teams do not check the output accurately. I was just working as external consultant on a training on AI, and I checked all the output and flagged the issues, but the internal team never reviewed it.
44/67
@alondigitized
the precedent is older than people remember — back in 2023, NY lawyer Steven Schwartz cited 6 ChatGPT-invented cases in Mata v. Avianca, got sanctioned $5k by Judge Castel. a top firm getting burned by the same playbook in 2026 is wild. full story:
The Lawyer Who Cited Fake Cases — The AI Files
45/67
@GregCook2011
I understand we stop referring to hallucinations and simply refer to errors as such
46/67
@BabyStarE8
When did shepardizing all of your cases go away for this firm? They have a much bigger problem than AI.
47/67
@QianjunBefanis
Happened in China too, law makers are caught quoted laws that does not exists, the entire nation shocked (and my guess: AI)
48/67
@AScotinExile
We in the UK are looking forward to the appeal in the Sandie Pegie case where the judge's summing up was a document full of similar issues. Popcorn time !
49/67
@Z17923667
Lolz they gets to be the example and this now is a cash cow for AI lawyer risk management firms… oh wait… time to create a new industry
50/67
@Bensam123TV
Been mentioning for awhile how it's weird that all the models, regardless of vendor, seem to regress around the same point. Like there is a overarching edict that controls the development of all of them and when humanity isn't ready they do stuff like this.
Can't blindly trust.
51/67
@J__Lory
Somebody remind me why their is a State Bar if they aren't going to reign in their profession?
@DAGToddBlanche
@HarmeetKDhillon
52/67
@Telemachanic
Apparently Westlaw hasn’t created its own reliable legal-only AI. Someone will soon. Then it’s light out.
53/67
@ClassicFord52
Interesting.
I am hopefully near the end of legal action against a Doctor in Mexico for malpractice that resulted in the death of my wife over four years ago.
Recently I decided to give AI a shot at telling me about other actions against this Doctor.
I passed the information along to my attorney, who used it to compare actual actions taking place against this Doctor, based on court filings and complaints to the medical board.
The two lists were not even close and of course I'm going to believe the verified information the attorney provided over AI.
54/67
@sailwriter2
Well you should see how my brother uses AI.
All results hallucinations as he thinks he is an AI ATTORNEY.
55/67
@devildriver6969
Hey @grok which AI did this?
56/67
@justwin3932
It would be so easy to submit the output into another AI and say verify these references
57/67
@PattontheDog1
You can’t spell “sanctioned” without AI.
58/67
@Cave__Cowboy
I like the potential of AI but it better get a lot better than this. Less A, more I.
59/67
@earl_kubin
I don’t believe this report. I’ve used AI for a while now and it may have biases I can’t prove but I would not go to school to be an attorney.
60/67
@HowardAulsbrook
That is what happens when you fail to verify the output. It amazes me that people believe you can prompt a model without worrying about the model's data being correct.
https://video.twimg.com/tweet_video/HGkr--4XgAAJ5Ky.mp4
61/67
@stillhumanno
Van Halen clause strikes again.
62/67
@KVM_SFR
sounds like a technology literacy issue. AI is the best entry-level associate you've ever had. properly trained and curated, it becomes an incredibly talented partner-level workhorse.
$2k/hr buys accountability and ownership of work-product.
63/67
@MarsBars90504
The internet will make everyone smarter they said.
AI will make people more efficient they said.
But nobody is tackling the lies, we have idiots screaming fake news, faked moon landing, flat earth anti-vax, chem trails, watching reality TV, believing in gods.
https://video.twimg.com/tweet_video/HGiwfJ1bUAAIhga.mp4
64/67
@lz4202
Top law schools that Sullivan and Cromwell hires from:
- Columbia
- Harvard
- Yale …
The associates prepping this case no doubt did exactly what they had been doing through Law school - and built their case on AI rather than doing any real work. And the partner was too lazy to check their work.
NEVER hire Ivy grads. These entitled mommies little dears are worse than useless.
65/67
@GonzoLiquidity
Wait for the next phase when AI primarily trains on AI output. Idiocracy would look like a reasonable place.
if you catch it. On some…
