causal
Self-Supervision Works Better on Redundant Data
Self-supervised prediction algorithms learn more structure when input data is more redundant; perceptual/sensory inputs like vision have far more redundancy than text (which is already compressed and less redundant), so self-supervision works better on perception than on language.
causalpending
Speaker
Yann LeCunEvidence Quote
“there is way more redundancy in the structure in perceptual inputs, sensory input like vision than there is in text, which is not nearly as redundant”
Created: 6/13/2026, 3:30:53 AM
Your notes
Loading notes...