normative
Contrastive Methods Should Yield to Regularized Methods
JEPA architectures can be trained with non-contrastive methods (distillation-based like BYOL and I-JEPA, or regularized like VICReg) rather than contrastive methods, avoiding the need for a gigantic number of negative contrastive samples when the answer space is large.
normativepending
Speaker
Yann LeCunEvidence Quote
“We have two sets of techniques. One set is based on distillation, and there’s a number of methods that use this principle, one by DeepMind called BYOL, a couple by FAIR, one called vcREG and another one called I-JEPA.”
Created: 6/13/2026, 3:30:53 AM
Your notes
Loading notes...