target encoder
A neural network component in JEPA that takes the target view (like masked blocks or a future segment) and produces the target embedding representing what is missing or happens next
As defined in What Is Yann LeCun Cooking? JEPA Explained Simply
Where it comes up · 1
Most important claims · 4 of 4Search all claims for “target encoder” →
- Representation collapse is a failure mode where both the context and target encoders in JEPA can learn to output the same constant embedding for all inputs, with the predictor simply outputting a constant embedding that always matches, resulting in extremely low training loss but learning absolutely nothing about the world.What Is Yann LeCun Cooking? JEPA Explained Simply
- A typical JEPA learning setup contains three components: a context encoder that takes visible content and produces an embedding, a target encoder that takes masked or future content and produces a target embedding, and a predictor that takes the context embedding and tries to produce the target embedding.What Is Yann LeCun Cooking? JEPA Explained Simply
- At inference, the target encoder is typically not needed; instead, the trained context encoder serves as a feature extractor that can be used for classification, retrieval, similarity search, or downstream supervised fine-tuning in a representation extraction application.What Is Yann LeCun Cooking? JEPA Explained Simply
- The first practical solution researchers used to prevent representation collapse was updating the target encoder with exponential moving average (EMA), where instead of both encoders learning freely, the target encoder is updated very slowly as a delayed version of the context encoder.What Is Yann LeCun Cooking? JEPA Explained Simply