factual

Backpropagation is just the chain rule of calculus

The 1986 Rumelhart-Hinton-Williams backpropagation paper showed that multi-layer neural networks can be trained using what turns out to be just the chain rule of calculus, which requires that the entire computational graph from output to input be differentiable—achieved by replacing the non-differentiable step threshold with a smooth sigmoid.

factualpending

Speaker

Anil Ananthaswamy

Evidence Quote

“it is essentially the back propagation algorithm, which shows you how you can train a multi-layer neural network using what turns out to be just the chain rule of calculus. It's an extraordinarily simple idea in retrospect”

Source

336 | Anil Ananthaswamy on the Mathematics of Neural Nets and AI— Sean Carroll's Mindscape
Created: 6/13/2026, 12:20:04 AM

Your notes

Loading notes...