Training JEPA architectures requires designing cost functions that produce low energy on training data but high energy outside the training manifold; there are only two classes of methods for this: (1) contrastive methods push down energy on real data and push up energy on synthetic negative samples, but they don't scale well to high dimensions and require many negative samples; (2) regularized methods use regularizers to minimize the volume of low-energy space, which scales better.
factualpending
Speaker
Yann LeCunEvidence Quote
“I only know two classes of methods for this one set of method is called contrastive...another set of method...called regularized method”
Created: 8/11/2026, 1:17:36 AM
Your notes
Loading notes...