causal

Progress requires improving everything at once

Reproducing LeCun's 1989 net showed that 33 years of algorithmic progress alone only halved the error; real gains required simultaneously improving data (10x the training set), compute, kernels, and algorithms—no single factor dominates, each contributes roughly equally, so future progress will likewise come from broad-based ~20% improvements across the stack.

causalpending

Speaker

Andrej Karpathy

Evidence Quote

“everything needs to improve across the board: architecture, optimizer, loss function. It also has improved across the board forever.”

Source

Andrej Karpathy — AGI is still a decade away— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:26:44 AM

Your notes

Loading notes...