The I-JEPA method prevents collapse through momentum-based weight updates—maintaining a second encoder whose weights are an exponential moving average of the primary encoder's weights, which unexpectedly prevents representation collapse, though the theoretical explanation remains incomplete.
factualpending
Speaker
Yann LeCunEvidence Quote
“that's what we set the weights of this encoder to. And that somehow prevents collapse. Why? We don't really know.”
Created: 8/12/2026, 10:03:11 PM
Your notes
Loading notes...