Self-supervised learning works by showing models text with missing words, removing words, and training the system to predict the missing words, which forces the network to learn representations that capture meaning, syntax, grammar, and internal structure of language that can then be applied to downstream tasks.

definitionpending

Speaker

Yan LeCun

Evidence Quote

“the way those large langage mods are trained is you show them a text then you remove some of the words... then we train a very large neural network to predict the words that are missing”

Source

AI: Grappling with a New Kind of Intelligence | World Science Festival— World Science Festival
Created: 8/10/2026, 10:49:40 PM

Your notes

Loading notes...