Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94

Double descent is a phenomenon where test error exhibits a non-monotonic U-shape as model size increases while data is fixed: performance improves as networks grow, then worsens sharply around the interpolation threshold (where model parameters equal training examples), then improves again as model size continues to grow, and this pattern occurs across virtually all practical deep learning systems without early stopping.

“when you increase the size of the neural network slowly and if you don't do early stopping that's a pretty important detail then when the neural network is really small you make it larger you get a very rapid increase in performance then you continue to make it large and at some point performance will get worse and it gets and and it gets the worst exactly at the point at which it achieves zero training error precisely zero training loss and then as you make it large it starts to get better again”

Hard benchmarks change the field's conversation: when deep learning produces undeniable results on hard problems, debate changes and skepticism shifts, making the availability of rigorous benchmarks central to how machine learning progresses.

“that with hard results you know if you got a heart if you have machine learning deep learning as a field is very fortunate because we have the ability to sometimes produce these unambiguous results and when they happen uh the debate changes the conversation changes”

The key breakthrough enabling deep learning's success in 2012 was not a new algorithmic invention but the combination of three factors: large amounts of supervised data from the internet, GPU compute becoming available, and the conviction among researchers like Hinton that scaling neural networks on big data would work when previous skeptics doubted it.

“the ideas were all there the thing that was missing was a lot of supervised data and a lot of compute once you have a lot of supervised data and a lot of compute then there is a third thing which is needed as well and that is conviction”

Large neural networks do not overfit quickly because stochastic gradient descent finds approximately the point with smallest norm in the high-dimensional subspace of solutions achieving zero training error, and this minimum-norm solution is provably insensitive to small random variations in the training data when dimensionality is high.

“you got a huge neural network you have a huge number of parameters and now let's pretend everything is linear then there is this big subspace where a neural network achieves zero error and sgd is going to find approximately the point that's right approximately the point with the smallest norm in that subspace okay and that can also be proven to be insensitive to the small randomness in the data when the dimensionality is high”

Trainability is an absolute constraint ('invariant we cannot violate') on neural network design; any architectural choice must preserve the ability to train networks from scratch and achieve useful performance with available compute, because without this constraint all other design considerations become meaningless.

“training comes first we've got our pillar which is the training pillar and now we are trying to contort our neural networks around the training pillar we got to stay trainable this is an invo this is an invariant we cannot violate”

James Martens' invention of the Hessian-free optimizer in 2010, which successfully trained a 10-layer neural network end-to-end without pre-training from scratch, was the pivotal moment that convinced Sutskever that deep neural networks were fundamentally powerful because a network with 10 layers is analogous to the human brain firing for 100 milliseconds, allowing perfect object recognition.

“the first moment in which i realized that deep neural networks are powerful was when james martens invented the hessian-free optimizer in 2010 and he trained a 10-layer neural network end-to-end without pre-training from scratch and when that happened i thought this is it”

AI will become increasingly impactful and powerful, making it important to think about system impacts before release, with staged release and cross-company discussion about risks being part of wise governance as the field matures from 'childhood' to 'maturity.'

“my take on this is that the field of ai has been in a state of childhood and now it's exiting that state and it's entering a state of maturity what that means is that ai is very successful and also very impactful and its impact is not only large but it's also growing and so for that reason it seems wise to start thinking about the impact of our systems before releasing them”

The most beautiful thing about deep learning is that it actually works—you start with small neural networks and backpropagation, which are inspired by the brain, and when you scale them up with large data, they produce capabilities that match brain functions, which is unbelievable given how simple the core ideas are.

“i think the most beautiful thing about deep learning is that it actually works and i mean it because you got these ideas you got the little neural network you got the back propagation algorithm and then you got some theories as to you know this is kind of like the brain so maybe if you make it large if you make the neural network lodge and you train it a lot of data then it will do the same function of the brain does and it turns out to be true that's crazy and now we just train these neural networks and you make them larger and they keep getting better and i find it unbelievable”

Empirical evidence is the primary source of conviction about deep learning's effectiveness, analogous to how evolution shows that evolutionary processes are effective at designing organisms without needing to understand the internal mechanisms, or like how physics requires both theory and experiments to validate predictions.

“empirical evidence is most of your sort of empirical evidence kind of convinces you it's like evolution is empirical it shows you that look this evolutionary process seems to be a good way to design organisms that survive in their environment but it doesn't really get you to the insides of how the whole thing works i think it's a good analogy is physics you know how you say hey let's do some physics calculation and come up with some new physics theory and make some prediction but then you gotta run the experiment you know you gotta run the experiment it's important so it's a bit the same here except that maybe some sometimes the experiment came before the theory but it still is the case you know you have some data and you come up with some prediction you say yeah let's make a big neural network let's train it and it's going to work much better than anything before it and it will in fact continue to get better as you make it larger and it turns out to be true”

Current AI skeptics often deny impressive AI capabilities by moving the goal posts and claiming the demonstrated task was not actually difficult, preventing genuine assessment of AI progress.

“Once you solve a problem it stops being hard and that's why you have a system that seemed impossible is now trivial and we adapt quickly and now there's uh sort of some cognitive scientists write articles saying that gpt2 models don't truly understand language.”

The transformer is the most important advance in neural network architectures in recent history, and its success comes from multiple simultaneous ideas, not just attention: transformers use attention, are designed to run efficiently on GPUs, and crucially are not recurrent (therefore more shallow and easier to optimize).

“the transformer is the most important advance in neural network architectures in recent history what is attention maybe too because i think that's the interesting idea not necessarily sort of technically speaking but the idea of attention versus maybe what recurring neural networks represent yeah so the thing is the transformer is a combination of multiple ideas simultaneously which attention is one do you think attention is the key no it's a key but it's not the key the transformer is successful because it is the simultaneous combination of multiple ideas and if you were to remove either idea it would be much less successful so the transformer uses a lot of attention but attention existed for a few years so that can't be the main innovation the transformer is designed in such a way that it runs really fast on the gpu and that makes a huge amount of difference this is one thing the second thing is the transformer is not recurrent and that is really important too because it is more shallow and therefore much easier to optimize”

There is probably an underlying evolutionary objective function (survive, procreate, ensure children succeed) from which other human drives emerge, but this doesn't answer the question of life's meaning.

“let me try to give the explanation maybe that will be that will work so you got a huge neural network let's suppose you've got a you are you have a huge neural network you have a huge number of parameters and now let's pretend everything is linear which is not let's just pretend then there is this big subspace where a neural network achieves zero error and sdgt is going to find approximately the point that's right approximately the point with the smallest norm in that subspace okay and that can also be proven to be insensitive to the small randomness in the data when the dimensionality is high but when the dimensionality of the data is equal to the dimensionality of the model then there is a one-to-one correspondence between all the data sets and the models so small changes in the data set actually lead to large changes in the model and that's why performance gets worse so this is the best explanation more or less so then it would be good for the model to have more parameters so to be bigger than the data that's right”

Aligning AGI to human values is a question about training a value function (objective/goal) that is learned separately from the agent's main capability function, similar to how humans have internal rather than external reward functions that they can choose to change.

“in some sense the kind of question which you are asking is so if you have to translate that question to today's terms yes it would be a question about how to get an rl agent that's optimizing a value function which itself is learned and if you look at humans humans are like that because the reward function the value function of humans is not external it is internal that's right and there are definite ideas of how to train a value function basically an objective you know and as objective as possible perception system that will be trained separately to recognize to internalize human judgments on different situations and then that component would then be integrated as the value as the base value function for some more capable rail system”

Self-play systems (OpenAI Five in Dota, hide-and-seek agents, AlphaZero) produce creative solutions that surprise researchers with truly novel strategies not anticipated during design, which is an important property for AGI that current supervised learning systems routinely lack.

“pretty much every self-play system both are dotabot i don't know if openai had a release about multi-agent where you had two little agents who were playing hide and seek and of course also alpha zero they were all surprising behaviors they all produce behaviors that we didn't expect they are creative solutions to problems”

Simulation is a tool with specific strengths and weaknesses that should be used strategically; transfer from simulation to the real world has been successfully demonstrated multiple times (e.g., OpenAI robot hand trained entirely in simulation and successfully transferred to physical world handling novel perturbations).

“transfer from simulation to the real world is definitely possible and has been exhibited many times in by many different groups it's been especially successful in vision also open ai in the summer has demonstrated a robot hand which was trained entirely in simulation in a certain way that allowed for cinderella transfer to occur”

Language models must be large to achieve semantic understanding because they must first model the surface-level patterns (characters, spacing, syntax), and only after exhausting syntactic patterns do they begin to capture semantics, and this progression from syntax to semantics with scale has been empirically demonstrated with the sentiment neuron discovery.

“when you don't know anything you'll notice very very broad stroke surface level patterns like sometimes there are characters and there is a space between those characters you'll notice this pattern and you'll notice that sometimes there is a comma and then the next character is a capital letter you'll notice that pattern eventually you may start to notice that there are certain words occur often you may notice that spellings are a thing you may notice syntax and when you get really good at all these you start to notice the semantics”

Reinforcement learning is neither vision nor language but naturally interfaces and integrates with both; its unique property is non-stationarity—as an agent's actions change, the world it observes changes in response, which is not the case for static supervised learning problems.

“what is unique about policy of learning to act well so one example for instance is that when you learn to act you are fundamentally in a non-stationary world because as your actions change the things you see start changing you you experience the world in a different way and this is not the case for the more traditional static problem where you have at least some distribution and you just apply a model to that distribution”

GPT-2 is a 1.5 billion parameter transformer trained on approximately 40 billion tokens of text from high-quality web pages (those linked from Reddit posts with more than three upvotes), and it achieved remarkable text generation quality that was surprising even to researchers familiar with prior progress in GANs and language modeling.

“gpt-2 is a transformer with one and a half billion parameters that was trained on upon about 40 billion tokens of text which were obtained from web pages that were linked to from reddit articles with more than three upvotes”

Backpropagation is a solution to the fundamental problem of finding a neural circuit subject to constraints, and this problem will not go away; therefore, it is unlikely that deep learning will move away from backpropagation in any dramatically different direction.

“i think it's a great algorithm because it solves an extremely fundamental problem which is finding a neural circuit subject to some constraints and i don't see that problem going away so that's why i i really i think it's pretty unlikely that we'll have anything which is going to be dramatically different it could happen but i wouldn't bet on it right now”