factual
Backpropagation is just the chain rule of calculus
The 1986 Rumelhart-Hinton-Williams backpropagation paper showed that multi-layer neural networks can be trained using what turns out to be just the chain rule of calculus, which requires that the entire computational graph from output to input be differentiable—achieved by replacing the non-differentiable step threshold with a smooth sigmoid.
factualpending
Speaker
Anil AnanthaswamyEvidence Quote
“it is essentially the back propagation algorithm, which shows you how you can train a multi-layer neural network using what turns out to be just the chain rule of calculus. It's an extraordinarily simple idea in retrospect”
Created: 6/13/2026, 12:20:04 AM
Your notes
Loading notes...