Co-Founder, @Gen_Legal_Inc Fmr: ML Engineer, @Harvard_Law, @CooleyLLP
Washington, D.C.
- Tweets 270
- Following 102
- Followers 842
- Likes 255
Will is one of the most prolific voices in legal tech today - and a voice that often dovetails with General Legal's fundamental worldview: you don't need to raise $500M to build great software for legal professionals.
Relatedly, selling billions of dollars of software to overpaid lawyers doesn't necessarily mean you will change anything about the delivery of legal services or the client's experience. You need to actually be incentivized to do that.
Thrilled and honored to be collaborating, @willchen500! More to come soon ;)
@MikeOSSTeam has been sponsored by @gen_legal_inc, a leading AI-native law firm backed by investors such as YC, Audacious Ventures and SV Angel, to develop a new open source project (Mike[???]) that will shake up the legal industry.
Currently in stealth.
More details to come.
Thanks @jaypmo @rwalk_xyz
5
3
67
10,177
Will is correct. Lawyers are only just scratching the surface of what’s possible. Most are using it as a better search tool. But we know so much more is possible if we use vibe coding as our guide. And I can see just how much more there is to go from the tools that we are building internally.
x.lingyaoai.com/willchen500/status/210…
This aligns with what I’m hearing from lawyers too. There has been an explosion in token spend recently due to models getting a lot smarter but lawyers are still using it mostly for very basic tasks like simple legal research, email drafting and summarisation, which to be fair is a lot of what lawyers do.
However, compared to how far agentic coding has come, it seems like we are still barely at the first inning in legal. It’s the equivalent of the summer of 2024 where programmers were still asking ChatGPT questions and copying and pasting snippets into their code base.
I think the reason for this is likely due to the following reasons in traditional law firms:
1. General inertia among the profession
2. Lack of awareness of AI capabilities
3. Limitations of harnesses used by them
This creates an opportunity for AI native firms to come in and gain market share by using AI wherever possible and feasible to automate what traditional law firms are still doing manually and passing on savings to clients via a flat fee structure.
4
25
2,586
A lot of folks have asked me: "why open source your core legal docs -isn't that your IP?" no. our exceptional lawyers are our IP. our massive deal flow is our IP. our precedent docs r like a programming language: underlying scaffolding to enable the work.
github.com/General-Legal/leg…
11
55
832
63,317
Fascinating finding by our benchmarks team... when we give models access to our knowledge base, the dumber ones actually get worse at legal lol
At General Legal, we use an internal knowledge system that our AI agents can refer to as they do their work. We've been testing various models on our internal evals recently and noticed something interesting: the smarter models benefit from the knowledge base, while the dumber models actually get worse. The correlation isn't perfect, but it seems that there is a threshold of intelligence required to properly use a knowledge base.
Notable that Astra seemed not to benefit that much. It was already scoring 90%+, so it's probable that the benchmark is saturated above that point--impossible to do much better. We'll need to make a harder eval. More on that later.
5
730
JP Mohler retweeted
I have been arguing this since early this year. Firms should try tools that actually work well instead of those with the shiniest marketing.
Unfortunately many firms who have already bought into certain vendors are reluctant to explore and implement new tools.
This is due to:
(1) institutional inertia caused by the billable hour that disincentivises actual productivity increase; and
(2) the amount of institutional weight that was put behind implementation of the tools the firms bought into. No one wants to admit that there has been no effective productivity increase (as is clear from the perspective of clients who have neither seen a change in billing structure nor the hours charged to them) and that some change is needed.
Was in another firm AI working group meeting recently. Partners. Associates. And engineers. All working to test and build AI workflows.
Firm has other software and AI like Legora. Teams were testing across models.
And Claude kept outperforming all the others.
Now yes some nuances here in favor of other tools like built in integration (to iManage for example) but those can be built too.
APIs to other sources and tools seem to be lagging across the board. Many legal specific software that dominate in a niche area do not offer APIs.
10
5
53
15,943
Existing AI benchmarks measure if a model completed a task. But at General Legal, we have the data to measure how AI performs against a real human lawyer.
We found that some models can hit close to 90% of the basic requirements. But in practice, they perform much, much worse than what our lawyers actually send clients.
You can read the full post below for all the details.
x.lingyaoai.com/gen_legal_inc/status/2…
We gave 13 AI models legal work to do from our new legal work benchmark. We compared the output of the models with what our human attorneys actually produced. We wanted to know whether a model could complete the whole assignment, not just how many individual requirements it could check off.
Key takeaways for our analysis:
- General Legal's new benchmark evaluates whether AI-drafted legal work is truly client-ready, not just technically complete.
- The benchmark tests agents legal work product with actual attorney work as the gold standard, including access to the firm's proprietary knowledge base.
- Legal work requires judgment about trade-offs, priorities, and strategic choices that extend beyond simply fulfilling a checklist of requirements.
- The benchmark tracks both individual changes and holistic quality, identifying when models add unnecessary terms or make choices attorneys wouldn't release to clients.
- All-pass rate measures the percentage of matters where output meets every criterion, reflecting that even one miss requires attorney intervention.
Read the full article for details.
1
2
266
The average lawyer who joins my AI-native law firm has 11 years of experience. Including 5+ years in Big Law.
I know the mainstream view is that startups are only for young people, but the reality is that not all the lawyers leaving Big Law for AI-native firms are just junior associates. Senior lawyers are leaving too.
AI-native firms aren't replacing sound human judgment with a commoditized software-based product. We are giving experienced lawyers better tools and, more importantly, a better business model.
x.lingyaoai.com/writeclimbrun/status/2…
I'm biased, but all these young lawyers (under 5 years) leaving biglaw to start AI native law firms seems to miss the point.
if I were a client, I don't need commodified legal work. I need judgement and many of these lawyers don't even have their training wheels off yet.
6
6
66
10,539
Much of the work AI automates is far less interesting than the term "craft" implies.
Industrialization of knowledge work is going to be as disruptive a shift as industrialization of physical labor was. As then, once craft fields are under pressure to produce volumes of standardized product while using less of the craft that made the work interesting & meaningful.
1
3
259
Newspaper revenue looked pretty good at the height of the dot com boom. Then it didn't.
Is the law firm business model broken? Biglaw has been among the best-performing areas of the economy over the last 20 years. When I quit Biglaw in 2010, PPP was $2.25–3m at high-end firms. Today, it's $5.5–11m!
Admittedly, there are challenges - not least bcs of AI - but, wow, the Biglaw business model has been waaaay better than I expected!
(That said, I am super enthusiastic about AI-first firms winning low & middle-end legal work, as well as vastly increasing the amount of legal work done)
Sorry for the Claude chart
2
10
1,756
