back propagation
A calculus-based learning algorithm (developed in the late 1970s) that adjusts a neural network's weights by propagating output errors backward through hidden layers to assign credit or blame.
As defined in 68 | Melanie Mitchell on Artificial Intelligence and the Challenge of Common Sense
Where it comes up · 5
- Possible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital3 claims
- Facing Superintelligence (with Ben Goertzel)2 claims
- How AI Is Changing Warfare | Palantir CTO1 claim
- Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton1 claim
- 68 | Melanie Mitchell on Artificial Intelligence and the Challenge of Common Sense1 claim
Most important claims · 8 of 8Search all claims for “back propagation” →
- Backpropagation, developed in the late 1970s and based on calculus, solved the previously unsolved problem of how to apportion credit and blame for output errors back through hidden layers, enabling the training of multi-layer neural networks and disproving Minsky and Papert's doubt that such an algorithm could exist.68 | Melanie Mitchell on Artificial Intelligence and the Challenge of Common Sense
- Simulated neural networks are fundamentally unlike conventional computer software: with software a programmer writes lines of code whose intent can be inspected, whereas with neural nets the only code written tells the network how to learn (backpropagation), after which the network learns purely from data, so you don't know what it has learned until you ask it.Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton
- Back propagation is a superior learning algorithm compared to biological brain learning because it can pack vastly more information into fewer connections—large language models with 1 trillion connections contain roughly a thousand times as much knowledge as humans, who have 100 trillion connectionsPossible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital
- Back propagation works by computing how each weight in a neural network should change to reduce the discrepancy between desired output and actual output, then changing weights in those directions; the algorithm works backwards through the network to assign credit to each feature detector for helping or hindering correct predictionsPossible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital
- When Goertzel was teaching deep neural networks at University of Western Australia in the mid-1990s, training a network with 35 neurons using recurrent backpropagation took three hours on a fast Sun workstation, which prevented refinement and testing of ideas that are now practicalFacing Superintelligence (with Ben Goertzel)
- Forward deployed engineering was developed because conventional software companies use sales as the feedback loop (can we sell it?), while the true test is whether the software delivers the outcome in the field with the end user; so Palantir sends engineers who solve backward from the operator's problem and build software through a form of back propagation.How AI Is Changing Warfare | Palantir CTO
- A high percentage of historical AI paradigms—including genetic algorithms, logic-based AI, and hypervectors—would work effectively if scaled up with modern compute and data, but were abandoned due to perceived fundamental limitations that were actually resource constraintsFacing Superintelligence (with Ben Goertzel)
- Hinton developed backpropagation with colleagues in the 1980s, and it is fundamental to all modern deep learningPossible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital