normative

Eval Suites Calibrate Each New Model Release

Tobi converts tasks that current models fail into evals, then reruns them on every new model to see which capabilities have changed, calibrating his expectations from there.

normativepending

Speaker

Tobi

Evidence Quote

“I turn them usually into something called evals, just so every time I get a new model, I run it against the evals of stuff that I couldn't get them to do before”

Source

Tobi Lütke - Building Islands of Innovation - [Invest Like the Best, EP.394]— Invest Like the Best
Created: 11/11/2026, 2:07:18 PM

Your notes

Loading notes...