causal
Progress requires improving everything at once
Reproducing LeCun's 1989 net showed that 33 years of algorithmic progress alone only halved the error; real gains required simultaneously improving data (10x the training set), compute, kernels, and algorithms—no single factor dominates, each contributes roughly equally, so future progress will likewise come from broad-based ~20% improvements across the stack.
causalpending
Speaker
Andrej KarpathyEvidence Quote
“everything needs to improve across the board: architecture, optimizer, loss function. It also has improved across the board forever.”
Created: 6/13/2026, 3:26:44 AM
Your notes
Loading notes...