
Why Vertical LLM Agents Are The New $1 Billion SaaS Opportunities
What this covers
This Lightcone episode from Y Combinator (YC)) is a conversation between host Gary and Jake Heiler, the lawyer-turned-founder of Casetext. It works as a case study, starting from the company's decade before modern LLMs. An early user-generated-content model, in which lawyers would annotate case law for free, failed because billable-hour lawyers have little spare time. Even once revenue arrived, customers resisted new features, since greater efficiency cut into hourly billing. The public release of ChatGPT changed that: clients went from "don't change anything" to "we need to get ahead of this."
The middle of the conversation covers the pivot. Casetext tested an early version of GPT-4 with customers under an NDA (non-disclosure agreement)) with OpenAI. Heiler describes a day's work finishing in 90 seconds, and says he committed the whole 120-person company to Co-Counsel within 48 hours. He cites the jump from GPT-3.5's bar exam score at the 10th percentile to GPT-4's at the 90th. The launch showed the product-market fit signals Andreessen's checklist describes, and the company sold to Thomson Reuters shortly after.
The most practical stretch concerns engineering. Heiler explains how Co-Counsel's "skills" chain prompts into workflows modeled on how a top attorney would produce a research memo, with pass/fail criteria at each step, which makes test-driven development) possible at the prompt level. He also lists the unglamorous work before the model is called: OCR tuning, document-management integrations, proprietary datasets, and query decomposition. Gary argues that this is where moats form, and he rejects the "GPT wrapper" and "hallucination" dismissals, a position the episode itself flags as contested. Two further claims are also contested or speculative: that automated legal work will leave lawyers with more interesting jobs, and that o1 might be prompted with expert reasoning, not just correct answers.
Jake Heiler demonstrates that building successful vertical AI agents requires deep domain expertise, rigorous test-driven prompt engineering with evals, and solving the last mile from 70% to 100% accuracy—not merely wrapping LLMs—as exemplified by CaseText's acquisition for $650M two months after launching Co-Counsel on GPT-4.
- CaseText spent 10 years building legal domain knowledge and customer relationships before LLMs; when GPT-4 arrived, that foundation let them reach product-market fit immediately while others remained stuck at early LLM integration stages
- Test-driven development applied to prompting—writing gold-standard outputs for each step and iterating until 1,199/1,200 test passes—is essential because LLMs are unpredictable and lawyers require near-perfect accuracy; this IP-building process is not a 'GPT wrapper'
- The last mile from 70% viable demos to 100% production-grade systems requires domain-specific integrations (OCR tuning, document management system connections, proprietary datasets, search query decomposition), not just clever prompting—this is where defensible moats and pricing power emerge
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Even after CaseText had made real revenue and had satisfied customers making incremental improvements, they had difficulty convincing customers to adopt new features because lawyers earning $5M+ per year were reluctant to change working practices due to perverse incentives: being more efficient would reduce billable hours and thus compensation
“when there's just an incremental Improvement it's actually pretty easy to ignore a lot of our clients they never say this literally but you kind this impression you walk into the room their office and you try to pitch them a product and you say this is going to change everything about the way you practice and they go well I make $5 million a year I don't want nothing to change this technology PL it's not I do not want to introduce anything that has the opportunity to make my life at all worse um or potentially worse or potentially more efficient because they build by the hour”
CaseText initially tried to build a user-generated-content (UGC) model similar to Wikipedia and Stack Overflow where lawyers would annotate case law for free, but this failed because lawyers bill by the hour and have high opportunity costs, unlike Wikipedia editors who have time they want to spend altruistically
“we spent like a couple of years trying to get as Gary said lawyers to annotate case law and to provide information so it's like a ugc site like user generated that was a big focus of ours like the kind of one-punch of better technology but also better content um we you know at the time our heroes were like stock overflow and Wikipedia and GitHub and other kind of Open Source or ugc kind of websites and it was a total failure like we could not get lawyers to contribute their time and information”
The public release of ChatGPT shifted customer perception of AI from 'incremental tool' to 'existential threat,' immediately changing client behavior from 'don't change anything' to 'we need to get ahead of this'
“the same you know guys and gals were telling us I make $5 million a year why would it change anything about my life we like I make $5 million a year this is going to change something I need to be ahead of this the technology itself and we'll get into a second really changed what we can build for life employers but also the market perceptions of what was like what was necessary really Chang as well”
When Jake Heiler first encountered GPT-4 under NDA from OpenAI, he made the decision within 48 hours to pivot the entire 120-person company to build Co-Counsel, shifting everyone from their existing product roadmap
“it took maybe 48 hours for us to decide to take every single person of the company and shift what they were working on from what the projects we were then working on at the time to 100% of the company all working on building this new product we call co-counsel based on the Jeep D4 technology”
Before ChatGPT's public release, CaseText had early customer testing of GPT-4 under extended NDA with OpenAI, allowing law firms to experience the AI for the first time without knowing it was GPT-4, and they observed tasks that normally took a full day taking only 90 seconds
“they did let us extend the NDA to a handful of our customers and so we started having our customers use it and so you know for months before gb4 was launched publicly we had a number of law firms un like they had no idea they're using GPT for but they were like seeing something really special”
CaseText went from $0 to $100 million valuation in the first 10 years, then to a $650 million exit to Thomson Reuters just two months after launching Co-Counsel on GPT-4
“in the first 10 years you went from 0 to $100 million valuation and then in a matter of 2 months after the release of GPT 4 that valuation went to a liquid exit to Thompson Reuters for $650 million”
All successful SaaS businesses historically have been built around custom business logic and integrations with esoteric databases and systems (e.g., Salesforce was essentially a SQL wrapper around CRM databases); AI-based SaaS follows the same pattern
“it's like basically all SAS for a while was just like a SQL rapper right like if you think about like very successful companies like Salesforce they built that business logic around basically just databases and connections between like tables and a database”
The lessons from CaseText's AI engineering approach are directly applicable to other domains beyond law where high accuracy is critical, and founders should not abandon AI projects based on accusations of hallucination—instead, they should apply rigorous testing and domain-specific engineering to reach production accuracy
“I just want to encourage people to not kind of give up based on those tropes right like oh it hallucinates too much it's too an accurate it's do whatever there's for an example of anything it's like there's a path and you can do it”
Jake Heiler was a lawyer before founding CaseText, giving him firsthand experience with legal work inefficiencies and motivating the founding mission to 'bring the best of technology to the legal space'
“I as a lawyer I actually like the job a lot the parts of my job that I hated the most was when I had to interact with the technology that lawyers have to use um regularly to get the job done”
During the pre-ChatGPT, pre-LLM era, CaseText built incremental improvements on legal research workflows that were relatively easy for customers to ignore because lawyers billing by the hour were not motivated to change processes that worked, and did not perceive existential risk from the technology
“when there's just an incremental Improvement it's actually pretty easy to ignore a lot of our clients they never say this literally but you kind this impression you walk into the room their office and you try to pitch them a product and you say this is going to change everything about the way you practice and they go well I make $5 million a year I don't want nothing to change this technology PL it's not I do not want to introduce anything that has the opportunity to make my life at all worse um or potentially worse or potentially more efficient because they build by the hour”
Co-Counsel's core concept was an AI legal assistant that acts like a new team member: it can read millions of documents to detect fraud, summarize large document sets, perform legal research, and generate memos citing specific cases—all within hours instead of weeks of attorney labor
“the idea we came up with again like 48 hours like a weekend after seeing gp4 was um and it's something that not but kind of still sound crazy today but it's feel felt crazy at the time which is this AI Legal Assistant by which we mean it's like almost like a new member of the firm you can just talk to it um not unlike how you might talk to something like chat GPT today uh and give it tasks like I need you to read these a million documents for me and tell me if there's any evidence of fraud happening in this company and then within a couple of hours it's like I've read all the documents here's what the summary is or summarize documents or do legal research and put together a whole memo after researching you know hundreds or thousands of cases answering the lawyer's initial research question”
Co-Counsel launched with real product-market fit signals: servers going down, inability to hire support and sales staff fast enough, major press coverage (CNN, MSNBC, ABA Journal), matching Mark Andreessen's checklist for true product-market fit from his 2005 article
“when we launched Co counil it was literally exactly that our servers were going down we could not hire support people fast enough we couldn't hire sales people fast enough I ate a lot of bucks you know um before we was a really big day if we in the ABA journal or some other you know legal specific uh publication we were on CNN and MSNBC and like you know all of a sudden everything changed and that's real product Market fit looks like”
Many companies dismiss LLMs as 'just wrappers' that 'hallucinate too much' for real-world use, but this is a tired trope disconnected from what successful startups are actually building—there's a clear path to production accuracy through rigorous engineering, not just fine-tuning or trivial prompt changes
“there's a lot of uh sort of the naysay are saying that a lot of companies are just building GP rappers and there's not a lot of Ip getting built but it's actually there's a lot of finesse to how you explain all of this like can you tell us about all of that and how much more there's to be built”
Jake also had computer science training in addition to his legal background, which was unusual and allowed him to identify technological solutions while still deeply understanding legal workflows
“and what was interesting about your is you also happened to be the rare breed of having also computer science training so this must have driven you nuts yeah exactly”
There are 'dozens' of vertical AI agent companies being founded in YC's latest batches, and CaseText is the most successful example to date in terms of scale of deployment and acquisition value
“it's a lot of vertical AI agents I mean I was trying to count the ones in s24 we have literally dozens of the YC companies in the last batch for building vertical specific AI agents and I think Jake is the founder who is currently running the most successful vertical AI agent it's by far the largest acquisition and it's actually deployed at scale in a lot of mission critical situations”
GPT-3.5 scored at the 10th percentile on the bar exam (barely passing, equivalent to random guessing by some test-takers), while early access GPT-4 scored at the 90th percentile on a new, non-training-set bar exam, representing a massive leap in legal reasoning capability
“one one like totem or like one example along the way is when gbd 3.5 came out the study was run um and it showed that gbd 3.5 got a 10 10th percentile on the bar passage right so like it did better than some people actually but the 10% of them yeah probably the ones who just filling out randomly basically um when we got early X gd4 we're like let's run the study again too and we work with open AI we're like we going to confirm this this test is not in the training set and it wasn't totally new test to it and the test we ran it did better than 90% of the test takers right”
Earlier LLM versions (GPT-2, 3.5) were 'making stuff up wildly' and hallucinating, making it 'very hard' to connect them to real legal use cases where 'you can't even make wrong assumptions' because legal work has high stakes for accuracy and factual correctness
“every time we're like no this sucks like you know by by time we got to three and 3.5 it was like okay well this is plausible sounding English and sounds kind of like a lawyer so kudos for that but it is just making stuff up wildly like we just didn't it's very hard to connect it to a real use case especially in legal where it's so important that you actually get the facts right the you can't hallucinate um you can't even you know make the wrong kinds of assumptions”
CaseText called the multi-step workflows built for Co-Counsel 'skills,' each composed of multiple prompts chained together, with each step having clear pass/fail criteria; this architecture allowed for test-driven development at the prompt level
“with the things that we built for coals we call them skills which felt very unique in at the time I think a lot of companies Now call their AI capability skills so when you're building these skills um it turns out it usually takes a lot of work to go from like say the customer inputs something say like a set of documents or a question or what have you to the end result that they're looking for”
The jobs that AI agents replace in domains like law will not disappear; rather, they will become more interesting because lawyers will shift from document review and basic research to strategic and intelligent work
“and there's some good news in that uh you know what the jobs aren't going to go away they'll just be more interesting that's what I think”
Building production-grade LLM applications requires integrating dozens of layers of domain-specific logic before the LLM is even called: proprietary datasets, OCR tuning (handling handwriting, page tilting, four-pages-per-sheet printing), document management system integrations, search query decomposition with SQL-like syntax, and other esoteric niche integrations specific to the vertical
“when you're actually trying to solve a problem for a customer and actually doing the job in our case of like what a young associate it might do and do it really well there are many layers of things you have to add in to actually get the job done and by the time you like add that all up you're not like a GPT rapper you're a full application that may include in our case proprietary data sets like the law itself and our annotations to the law that we added automatically it may include um connections into customer databases in our case in legal they have these very specific legal specific document Management Systems um you know so connecting into those is like very important”
Solving the 'last mile' from 70% viable demos to 100% production-grade systems is fundamentally different in effort and nature from creating the initial LLM prototype, and this is where defensible business value and pricing power emerges
“bridging that gap between um something that like either a very technical person can do but most people can't and making accessible or um bridging that gap between that almost works like you can do a lot of cool Demos in chat GPT without building a line of code but that almost works and works you know 70% of the time but going to 100% of the time is a very different kind of task and people will pay $20 a month for the 70% and maybe $500 or $1,000 a month for something that actually works depending on the use case right so there's a lot of value gained going that last mile or 100 miles whatever it is”
Building Co-Counsel required working backward from the desired output (a legal memo with citations to specific cases) and decomposing it into a multi-step workflow mimicking how the best attorney would solve the problem: break down English queries into search queries, execute searches against case databases, read and summarize results, compile citations, then write the final memo
“the process that we started with then it's actually not too dissimilar to what we're doing today it started with a question of like okay well what problem are we trying to solve for the user right the user wants to do research uh legal research um so and they want like a memo answering their question with citations to the original source so like that's the end result and then we're like well how do we go from that end result like working backwards almost what would it take to get there”
A promising new prompting technique with o1 is teaching it not just the answer, but how to think through the problem, potentially by examples of expert reasoning that go beyond just gold-standard outputs
“there's a chance that that one of the new prompting techniques with L1 is teaching it not just like how to answer the question what examples of good answers look like but how to think and I think that that's another like really interesting opportunity here is is um injecting domain expertise or um just your own intelligence”
CaseText is investigating whether it can prompt o1 to be explicit about what it thinks about during its reasoning process, and then inject domain expertise by telling it 'here's how the best lawyers in the country would think about solving this problem,' potentially amplifying reasoning capability
“one of the things that we're investigating for what it's worth with o1 is can we prompt it to tell it what to think about during its thinking process and inject like again like we've hired some of the best lawyers in the country how would the some of the best lawyers in the country think about solving this problem and maybe you know we have no conclusive evidence one way or the other yet that this dramatically improves things is so early uh and just just not enough time yet has passed”
During the months before GPT-4's public launch, CaseText's entire 120-person company worked in deep focus with minimal sleep, iterating rapidly on Co-Counsel because they perceived they had a months-long head start before competitors could access the same technology
“the whole company all 120 of us did not sleep for those you know months before gbd4 was like publicly launch therefore could publicly launch the product we felt like we had this amazing opportunity to run far ahead of the market something really beautiful happens when everybody's working super super hard which is you iterate so quickly past and actually I I still see some companies out there they're stuck where we were in the first month of seeing gbd4 right um and I think it's because they're just not like as intensely focused and engaged as as we were were able to be during those like couple like about 6 months or so before the public launch of gbd4”
For each step in a skill, CaseText wrote gold-standard test cases defining what good output looks like, starting with dozens of tests and scaling to thousands, then iterating prompts until achieving ~99%+ accuracy (e.g., 1,199 out of 1,200 correct) on the test set
“we had a Ser series like a battery of tests before but this got way more intense where we' write at first maybe a few dozen tests and then a few hundred and a few thousand for every single one of those PRS so you know if if the the the job to be done in the very beginning of this research process for example is taking the English language query and breaking it down into search queries we had a a very clear sense of what good search queries look like and wrote like a gold standard answers for giving this input this is what the outpa looks like right”
Early law firm customers using Co-Counsel under NDA before ChatGPT's public release experienced it as their first-ever interaction with a 'Godlike' AI, observed tasks that previously took a full day being completed in 90 seconds, and some customers expressed anxiety about their job security ('I should retire soon')
“this is their first ever experience talking to this Godlike feeling you know AI that was all of a sudden doing these tasks that would take me when I practiced like a whole day and it's being done in a minute and a half”
Jake personally built the first prototype version of Co-Counsel himself in the first 48 hours (with initial NDA restricted to him and his co-founder), building it over a weekend before telling the company a week and a half later, to overcome internal skepticism and lead by example
“I led the way through example I built the first version of it myself um wow even with 120 person company with like a whole bunch of Engineers and lawyers and stuff like mhm before that you like opened up your like IDE and actually built the thing yourself oh yeah and part of it was uh the NDA only extended at first to me and my co-founder that was it that was a blessing though it it turned out to be like perfect”
Jake brought in customers early (within the NDA window) to demonstrate real product impact, which converted skeptical internal stakeholders (sales, marketing, engineers) faster than any internal argument could, because seeing a customer's face light up with existential realization was more persuasive than data
“we also brought in customers early and that helped convince a lot of people as soon as like a skeptical sales or marketing or whatever person or even an engineer was on the other line end of a zoom call uh where um a customer was was reacting to the product in real time and giving us their honest reactions and like seeing the look on their face again you have to imagine it's almost hard to imagine that the world was like pre- chat GPT but then there some of these people were seeing that that exact idea for the first time and they were they were just blown away and that really changed Minds quickly I mean we saw people go through like existential crisises live you know on Zoom calls like oh my God you see their expression change exactly”
Jake did not believe in test-driven development before working with prompts, but now believes it's 10x more critical for LLM-based systems because models are unpredictable and adding instructions to fix one test case can break others, requiring constant, systematic validation
“I never really believed in test driven development before prompting like I was like the code works it doesn't it's fine like you'll see it when you but like with prompting actually think becomes even more important because of the kind of like nature of these llms they might go in Crazy directions unexpectedly and so you know you might very easily add in a set of instructions to solve one problem you're seeing with these sets of tests and then to break something with these sets of tests so so that that exact kind of theory of test development applies you know 10x more I'd say in the world of prompting”
One bad customer experience, especially a first bad experience, can permanently damage trust in AI tools, particularly for busy lawyers who are not technologists and may not return to a product for months or years after losing faith
“you can lose faith in these things really quickly right you have one bad experience especially if it's your first bad your first experience is bad and you're like you know maybe I'll check on this AI stuff a year from now especially if you're like a busy lawyer not a technologist so we we knew you had to make that first encounter the first week really really work for the lawyer or else they're not going to invest in it deeply”
Test-driven development with evals became obvious to Jake immediately, not based on general software engineering principles, but because he had 10 years of lawyer feedback saying 'the moment you make a mistake, we tell you immediately,' creating pressure to achieve near-perfect accuracy from day one
“I think I think the the biggest thing first of all depends on the use case for a lot of things that we were working on For Better or For Worse there was was a right answer and if you get the wrong answer lawyers are not going to be happy about it you know I had been a lawyer myself but also been s lawyers for a decade and every time we made the smallest mistake in anything that we did we heard about it immediately right and so I had that voice in my head maybe as I was going through this process”
Jake faced significant employee resistance to pivoting entirely to the Co-Counsel GPT-4 project, particularly from go-to-market staff who were experiencing 70-80% year-over-year growth with $15-20M ARR, and from board members who questioned why successful product should be abandoned
“this is especially true you running a business for 10 years because they've seen you wander through that Maze and and bump it to dead ends and a lot of those folks have been there for uh most or all that time watching you know me as the founder saying we're definitely going this direction it's definitely going to work and sometimes it doesn't and you only get so many of those with employees right so this was maybe my last one that I had with some of these folks and they're like here Jake goes again with this crazy new technology and some idea we're gonna invest deeply in”
In 2012, as a lawyer, Jake found it trivial to use Google and find movie times or nearby Thai restaurants on his new iPhone, but extremely difficult (requiring 5+ days of 5am work sessions) to find the specific evidence that would exonerate a client or win a billion-dollar lawsuit, signaling that legal information retrieval was a solved problem for entertainment but unsolved for high-stakes legal work
“I remember thinking and this is like 2012 when I was at a law firm if I want want to do something really trivial I had like a new iPhone at the time I can go on Google and find like movie times or where's the closest open Thai restaurant with vegetarian options that was super easy but if I wanted to find the piece of evidence that was going to exonerate my client and and make it so he doesn't have to go to jail for the rest of his life or the um key legal case that will help me win a billion dollar lawsuit well that's going to be like 5 days in a row till 5:00 a.m. every day it's like there's got to be a better way”
o1's improved performance may be attributable to OpenAI training it on examples of human internal monologue or thinking-out-loud processes, where contractors were asked to articulate how they would solve problems step-by-step rather than just input-answer pairs
“I think there's a good shot that that they've had you know maybe change what their contractors are doing instead of just doing you know input in answer out they were doing input in how would I think about solving this problem and then answer out”
OpenAI o1 is a fundamentally different model that exhibits precise thinking capabilities, passing tests that earlier LLMs consistently failed: specifically, detecting subtle quote misquotations in a 40-page legal brief where the modification was as small as changing 'and' to 'neither nor'
“so first of all I think 's a very impressive model um like with other things we gave it the kinds of tests that we knew were failing and the degree of it's not just math degree of thoroughness precision intelligence applied to some of these questions and sometimes it's the stuff that you wouldn't wouldn't expect you need a super smart model to do like in one of the tests that we run we give it uh lawyers real legal brief but we edited very slightly some of uh that lawyer's quotations to the case to make it a wrong quotation or wrong kind of summarization of his case so he this like 40 page legal brief you alter things with just adding the word like not can change the meaning of something entirely right and then we give the full text of the case as well to the AI and we say well what did you know what did the lawyer uh get wrong about this case of anything and literally every llm before that would be like nothing it's perfectly right and it's just not a precise thinker about some of the the very nuanced things that we altered about the brief to make it slightly wrong and then one got gets it like immediately”
Many founders in the AI space are tempted to 'raw dog' prompting without evaluations or test-driven development, relying on 'vibes only,' which is a critical mistake because this approach provides no signal on why the model is failing or how to improve it
“one of the things that strikes me that is tricky like many Founders we work with are very tempted to just raw dog it yeah there's like no evales no test driven we're just like Vibes only prompt engineering”
CaseText must make the first week of lawyer interaction with Co-Counsel work really well, or the lawyer won't invest deeply in the tool—this high-stakes first impression requirement directly motivated the test-driven development and high-accuracy standards
“so we we knew you had to make that first encounter the first week really really work for the lawyer or else they're not going to invest in it deeply”
At his law firm, Jake built browser plugins on top of legal research tools to make his own workflow more efficient, which got him in trouble with the general counsel who claimed the law firm owned the technology and made him question whether to stay in law practice
“I was building like browser plugins to go on top of the the tools I was using just to make my like life more efficient and effective and actually one of the reasons I left the law firm to start a company and apply to YC was I got in trouble with the general counsel who thought like hey why are you spending all your time you know doing this Tech stuff and also made at the time very clear that that my Law Firm owns all that technology”
Jake originally created CaseText 11-12 years ago (approximately 2011-2012), spent the first 10 years in pre-LLM development, then pivoted entirely to Co-Counsel on GPT-4, which launched publicly a few months before the acquisition conversation started
“Jake heler of case text I think of Jake is a little bit like one of the first people on the surface of the Moon he created uh case tax more than I think 11 12 years ago actually and in the first 10 years you went from0 to $100 million valuation”
The acquisition conversations began just two months after Co-Counsel's public launch, and the transaction closed six months later
“to be clear the transaction only closed six months after we launched but it was two months the conversation started”
Before the internet and web-based tools, lawyers literally went to libraries and read books to research case law, and early web-based legal research tools were clunky and didn't allow basic features like Ctrl+F
“slightly before my time you'd literally go to the library and open up books and just start reading and you know new products were coming out that were some of the first web-based research tools but they were pretty clunky it was just hard to find the relevant information you couldn't do contrl F or any of this stuff basically basally not”
The idea that mental models and frameworks rhyme with breaking down complex problems into steps—Jake's Co-Counsel approach of decomposing legal research into 12+ discrete search queries, summarization, and citation steps rhymes with the test-driven development approach of breaking code into testable units
“I guess uh the mental model I have for you is there's this concept of the idea maze you know the founder goes in the beginning of the Maze and they're just like feeling around like actually uh in the arena talking to you know customers learning like where are the walls which which path to go should I go left or right”