For each step in a skill, CaseText wrote gold-standard test cases defining what good output looks like, starting with dozens of tests and scaling to thousands, then iterating prompts until achieving ~99%+ accuracy (e.g., 1,199 out of 1,200 correct) on the test set
factualpending
Speaker
Jake HeilerEvidence Quote
“we had a Ser series like a battery of tests before but this got way more intense where we' write at first maybe a few dozen tests and then a few hundred and a few thousand for every single one of those PRS”
Created: 8/10/2026, 10:57:58 PM
Your notes
Loading notes...