causal

Self-Supervision Works Better on Redundant Data

Self-supervised prediction algorithms learn more structure when input data is more redundant; perceptual/sensory inputs like vision have far more redundancy than text (which is already compressed and less redundant), so self-supervision works better on perception than on language.

causalpending

Speaker

Yann LeCun

Evidence Quote

“there is way more redundancy in the structure in perceptual inputs, sensory input like vision than there is in text, which is not nearly as redundant”

Source

Research Pioneer— Lex Fridman Podcast
Created: 6/13/2026, 3:30:53 AM

Your notes

Loading notes...