Backpropagation requires an exact model of forward computation to work, but analog hardware systems won't have good models of their own nonlinear properties, making it extremely difficult to scale backpropagation in analog systems; smaller versions work on CIFAR-10 but not on ImageNet scale problems.
causalpending
Speaker
Jeff HintonEvidence Quote
“if you think about how back propagation works you have an exact model of the forward computation and that's why you can do back propagation in this analog Hardware the system itself won't have a good model of its properties is and so it seems very hard to do back propagation... people can make it work on c510 but they can't make it work on image net”
Source
Geoff Hinton - Will Digital Intelligence Replace Biological Intelligence? | Vector's Remarkable 2024— Vector InstituteCreated: 8/11/2026, 6:32:35 AM
Your notes
Loading notes...