Collapse Prevention in Representation Learning

Definition

Problem in self-supervised learning where encoders learn trivial solutions (e.g., outputting constants regardless of input) that minimize loss but produce useless representations; two solution families exist: (1) contrastive methods that explicitly push up energy on negative samples, and (2) regularized methods that constrain the volume of low-energy space.

Cast within

No topic-region cast yet — this appears once Collapse Prevention in Representation Learning's compiled claims are aligned into a topic region's argument tree.

Claims mentioning Collapse Prevention in Representation Learning (2)

Your notes

Loading notes...