Inference
Filling in the blanks between what an author explicitly said and what the author meant, drawing on background knowledge.
As defined in Read Like a Champion (with Doug Lemov)
Other definitions
The process of using a trained AI model to make predictions or generate outputs on new inputs at runtime, as opposed to the training phase where the model learns parameters.
Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner
Where it comes up · 69
- Capital, Compute & AI Scaling11 claims
- Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!10 claims
- Dwarkesh Patel Podcast - Dylan Patel – Anthropic & OpenAI will have most of the world’s compute by 20286 claims
- Artificial General Intelligence's Five Hard National Security Problems6 claims
- Cloudflare: Leading Cybersecurity5 claims
- Week 1 of the Trump administration, AGI timelines, and DeepSeek5 claims
- AI Trends 2025: Mary Meeker Deck Deep Dive Part 14 claims
- Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner4 claims
and 61 more
Most important claims · 10 of 166Search all claims for “Inference” →
- Making inferences is not a transferable skill that can be taught generically and applied to any text; rather, inference is domain-specific and depends on the reader's background knowledge, so practicing 'making inferences' in the abstract is a waste of time compared to building rich background knowledge.Read Like a Champion (with Doug Lemov)
- Inference costs have fallen about 99 percent, following the same pattern as internet transit costs, which fell about 25 percent a year for 20 years while the total profit pool stayed static.Capital, Compute & AI Scaling
- Training is a constant, saturating exercise that uses compute at maximum utilization for months, whereas application inference is extraordinarily bursty, so the hardware, cluster design, and utilization needs for inference differ from those for training.Capital, Compute & AI Scaling
- Pre-training scaling laws are now being supplemented by post-training scaling, multimodality scaling, synthetic data generation, and inference-time scaling, with inference-time scaling now scaling up to achieve intelligence through reasoning, simulation, and reflection rather than just model size.Ep17. Welcome Jensen Huang | BG2 w/ Bill Gurley & Brad Gerstner
- The marginal cost of AI inference varies enormously by user: casual chatbot use costs about the same as serving a webpage, while users applying test-time scaling to long reasoning, such as solving a math theorem, incur costs that rise with every second of thought.Ben Thompson on Big Tech, China, and the AI Boom Running Out of Money - [Invest Like the Best, EP.487]
- Cloud computing is effectively a logistics business: providers must forecast demand and build supply on a multi-year basis, and the 2023 to 2024 surge in demand hit inelastic data center supply, but a predictable demand cadence turns it back into a methodical rollout.Capital, Compute & AI Scaling
- Optical fiber providers (Corning and Light) represent the next evolution in data center networking because copper wires become inefficient and generate excessive heat when used to connect very large numbers of GPUs, making optical fiber the necessary next-generation solution.Leopold Aschenbrenner says "No More Stocks!"
- When the cost of a computational resource falls, total demand rises more than proportionally, as the spreadsheet showed by creating vastly more accounting and modeling rather than eliminating accountants, so cheap intelligence will generate far more use.Gustav Söderström - How Spotify Thinks - [Invest Like the Best, EP.424]
- Unlike classic software, which amortizes a large upfront investment to near-zero marginal cost, AI carries high marginal cost per use, so AI businesses must cover inference costs through subscriptions or ads.Gustav Söderström - How Spotify Thinks - [Invest Like the Best, EP.424]
- Inference is memory-bound because each parameter must be loaded from memory and used once, so batching many users to reuse each loaded weight amortizes the expensive load, and a transformer ASIC can batch around 2,500 users versus about 64 on GPUs.Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]