normative
Eval Suites Calibrate Each New Model Release
Tobi converts tasks that current models fail into evals, then reruns them on every new model to see which capabilities have changed, calibrating his expectations from there.
normativepending
Speaker
TobiEvidence Quote
“I turn them usually into something called evals, just so every time I get a new model, I run it against the evals of stuff that I couldn't get them to do before”
Source
Tobi Lütke - Building Islands of Innovation - [Invest Like the Best, EP.394]— Invest Like the BestCreated: 11/11/2026, 2:07:18 PM
Your notes
Loading notes...