Forward Pass
A play in American football where the ball is thrown forward toward the opponent's goal line; rule changes in the late 1970s made it easier and central to the offense.
As defined in Matthew Futterman on Players and the Business of Sports
Other definitions
The computational process in neural networks where input data propagates through layers of the network to produce output predictions, contrasted with backward pass where gradients are computed for training
The Engineering Unlocks Behind DeepSeek | YC Decoded
Where it comes up · 7
- Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch3 claims
- Sholto Douglas & Trenton Bricken — How LLMs actually think3 claims
- This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)1 claim
- The Engineering Unlocks Behind DeepSeek | YC Decoded1 claim
- Gary Belsky on the Origins of Sports1 claim
- Matthew Futterman on Players and the Business of Sports1 claim
- 230 | Raphaël Millière on How Artificial Intelligence Thinks1 claim
Most important claims · 10 of 11Search all claims for “Forward Pass” →
- The NFL deliberately changed its rules in the late 1970s—allowing linemen to extend their arms when blocking and prohibiting contact with receivers beyond 5 yards—to open up the passing game and increase scoring, transforming the sport from a coach's league (Lombardi) to a quarterback's league (Montana).Matthew Futterman on Players and the Business of Sports
- The profound accomplishment of AlphaGo is that roughly ten sequential layers of neural-network computation can amortize and approximate to high fidelity a nearly intractable, NP-class search problem—suggesting our worst-case-complexity framing of hard problems may be incomplete, since hard problems often have exploitable structure.Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch
- Although language is theoretically infinitely recursive, empirically humans only handle five to seven levels of nested recursion, matching the magic number of items held in working memory, suggesting there is a finite number of useful forward passes—but you can compensate by adding more layers.Sholto Douglas & Trenton Bricken — How LLMs actually think
- Ascribing intrinsic goals or desires to current language models is likely a category mistake because they learn passively from a stream of text, never genuinely interacting with the world, and after training are 'frozen'—their weights no longer adjust—whereas intrinsic, dynamic goals plausibly require more active learning and ongoing weight-adjusting interaction.230 | Raphaël Millière on How Artificial Intelligence Thinks
- By distilling the MCTS-improved distribution back into the raw policy network, you amortize the work that thousands of search simulations do into a single forward pass, so the next round of self-play starts from a much stronger base and reaches higher win rates for the same simulation budget.Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch
- In-context learning can be viewed as very similar to gradient descent, with the attention operation acting like gradient descent on the in-context data—n steps of gradient descent match n layers of in-context learning, and loss on tasks like linear regression goes down with shots matched to gradient descent steps.Sholto Douglas & Trenton Bricken — How LLMs actually think
- Chain-of-thought doesn't just transmit the single output token; during the forward pass the model creates KV (key-value) values that future steps attend to, so the model can smush information about potential futures into the keys and values it might want, transmitting far more than the tiny log-vocab-size bits of one token.Sholto Douglas & Trenton Bricken — How LLMs actually think
- Although AlphaGo vs. Lee used tens of thousands of simulations per move on a TPU pod, modern Go bots need little test-time compute because the policy improvement progressively pushes the search burden into the raw network, so most work is done in one forward pass with search only adding extra strength.Dwarkesh Patel Podcast - Eric Jang – Building AlphaGo from scratch
- Deep Seek V3 uses a mixture of experts architecture with 671 billion total parameters but only activates 37 billion parameters for each token prediction, compared to Llama 3's largest model which uses 405 billion parameters without mixture of experts, meaning V3 activates 11x fewer parameters per forward pass.The Engineering Unlocks Behind DeepSeek | YC Decoded
- The forward pass in Colibri is validated to be token-exact against a reference transformers implementation, ensuring that the streaming and quantized version produces identical outputs to the full-precision version.This Tiny Engine Runs Impossibly Big AI Models Locally! (colibrì)