For each step in a skill, CaseText wrote gold-standard test cases defining what good output looks like, starting with dozens of tests and scaling to thousands, then iterating prompts until achieving ~99%+ accuracy (e.g., 1,199 out of 1,200 correct) on the test set

factualpending

Speaker

Jake Heiler

Evidence Quote

“we had a Ser series like a battery of tests before but this got way more intense where we' write at first maybe a few dozen tests and then a few hundred and a few thousand for every single one of those PRS”

Source

Why Vertical LLM Agents Are The New $1 Billion SaaS Opportunities— Y Combinator
Created: 8/10/2026, 10:57:58 PM

Your notes

Loading notes...