James Martens' invention of the Hessian-free optimizer in 2010, which successfully trained a 10-layer neural network end-to-end without pre-training from scratch, was the pivotal moment that convinced Sutskever that deep neural networks were fundamentally powerful because a network with 10 layers is analogous to the human brain firing for 100 milliseconds, allowing perfect object recognition.

factualpending

Speaker

Ilya Sutskever

Evidence Quote

“the first moment in which i realized that deep neural networks are powerful was when james martens invented the hessian-free optimizer in 2010 and he trained a 10-layer neural network end-to-end without pre-training from scratch and when that happened i thought this is it”

Source

Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94— Lex Fridman
Created: 8/10/2026, 10:49:52 PM

Your notes

Loading notes...