Collapse (in joint-embedding systems)
A training failure mode where the joint-embedding system learns a trivial solution by ignoring inputs and producing constant or meaningless representations, making the prediction error zero everywhere without learning anything useful
As defined in Yann LeCun: World Models: Enabling the next AI revolution
Where it comes up · 1
Most important claims · 1 of 1Search all claims for “Collapse (in joint-embedding systems)” →
- Joint-embedding systems can collapse by ignoring inputs and producing constant representations; preventing collapse is the central problem in self-supervised learning for these systems and requires either contrastive methods (pushing up energy of non-data points) or regularized methods (minimizing volume of low-energy regions)Yann LeCun: World Models: Enabling the next AI revolution