energy function
A function that measures the degree of incompatibility or 'energy' between observations and predicted outputs; lower energy values indicate compatibility while higher values indicate incompatibility.
As defined in The Shape of AI to Come! Yann LeCun at AI Action Summit 2025
Where it comes up · 1
Most important claims · 6 of 6Search all claims for “energy function” →
- Classical inference in statistics, structure prediction, and classical AI uses a different paradigm than LLMs: these domains use an energy function that measures the degree of incompatibility between observations and proposed outputs, and inference consists of finding the output that minimizes this energy/incompatibility—this is more akin to human System 2 thinking (deliberate reasoning about actions before taking them) than LLM System 1 (automatic pattern matching without reasoning).The Shape of AI to Come! Yann LeCun at AI Action Summit 2025
- Training a system to predict what will happen next in a video is a natural approach for learning world understanding because if a system can predict future video frames it presumably has learned the underlying structure of the world; however, this approach for video fails where it works for text: with text there are a finite number of possible next tokens so prediction can succeed, but with video there are infinite possible pixel configurations making exact frame prediction intractable.The Shape of AI to Come! Yann LeCun at AI Action Summit 2025
- Training JEPA architectures requires designing cost functions that produce low energy on training data but high energy outside the training manifold; there are only two classes of methods for this: (1) contrastive methods push down energy on real data and push up energy on synthetic negative samples, but they don't scale well to high dimensions and require many negative samples; (2) regularized methods use regularizers to minimize the volume of low-energy space, which scales better.The Shape of AI to Come! Yann LeCun at AI Action Summit 2025
- Energy-based models represent dependencies between variables (observations and outputs) through a function that takes low values when variables are compatible and high values when they are incompatible, avoiding the need to model full probability distributions.The Shape of AI to Come! Yann LeCun at AI Action Summit 2025
- Training joint embedding architectures requires ensuring that the energy function (divergence measure between predicted and target representations) is low on the training data manifold but high outside it, which can be achieved through two classes of methods: contrastive methods (explicitly pushing negative samples up) and regularized methods (constraining the manifold volume using regularizers like correlation or covariance penalties).The Shape of AI to Come! Yann LeCun at AI Action Summit 2025
- The correct inference model for AI systems should not be autoregressive token generation but rather optimization over an energy function that measures compatibility between observations and proposed outputs, allowing systems to spend more computation on harder problems (system two thinking) rather than spending identical computation on simple yes-or-no questions and complex questions.The Shape of AI to Come! Yann LeCun at AI Action Summit 2025