Self-supervised learning works by showing models text with missing words, removing words, and training the system to predict the missing words, which forces the network to learn representations that capture meaning, syntax, grammar, and internal structure of language that can then be applied to downstream tasks.
definitionpending
Speaker
Yan LeCunEvidence Quote
“the way those large langage mods are trained is you show them a text then you remove some of the words... then we train a very large neural network to predict the words that are missing”
Source
AI: Grappling with a New Kind of Intelligence | World Science Festival— World Science FestivalCreated: 8/10/2026, 10:49:40 PM
Your notes
Loading notes...