The cost function is a profound foundational idea in deep learning because it allows reasoning about system behavior through optimization, and it is unwise to bet against cost functions as the central organizing principle despite emergence of alternatives like GANs and self-play which use different mathematical objects (game equilibrium, self-competition) for reasoning about behavior.

normativepending

Speaker

Ilya Sutskever

Evidence Quote

“i'm a big fan of cost functions i think cost functions are great and they serve us really well and i think that whenever we can do things because with cost functions we should and you know maybe there is a chance that we will come up with some yet another profound way of looking at things that will involve cost functions in a less central way but i don't know i think cost functions are i mean i would not better guess against cost functions”

Source

Ilya Sutskever: Deep Learning | Lex Fridman Podcast #94— Lex Fridman
Created: 8/10/2026, 10:49:52 PM

Your notes

Loading notes...