The I-JEPA method prevents collapse through momentum-based weight updates—maintaining a second encoder whose weights are an exponential moving average of the primary encoder's weights, which unexpectedly prevents representation collapse, though the theoretical explanation remains incomplete.

factualpending

Speaker

Yann LeCun

Evidence Quote

“that's what we set the weights of this encoder to. And that somehow prevents collapse. Why? We don't really know.”

Source

Yann LeCun: Special Lecture on AI and World Models— Al-Khwarizmi Applied Mathematics Webinar
Created: 8/12/2026, 10:03:11 PM

Your notes

Loading notes...