Compute
The total amount of computational processing power (and the hardware/energy behind it) used to train or run an AI model.
As defined in et_995
Other definitions
Computational capacity or processing power; the amount of calculation a system can perform. Used here to mean the neural or computational resources available for searching over Turing machine specifications.
The Grand Theory of Intelligence – Gwern
Where it comes up · 32
- OpenAI Co-Founder: “I just had to Quit”13 claims
- AI Today and Tomorrow6 claims
- Artificial General Intelligence's Five Hard National Security Problems5 claims
- Jack Clark4 claims
- The Possibilities of AI [Entire Talk] - Sam Altman (OpenAI)4 claims
- Sam Altman and Greg Brockman on AI and the Future (Full Audio)4 claims
- Leopold Aschenbrenner says "No More Stocks!"3 claims
- Ilya Sutskever (OpenAI Chief Scientist) — Why next-token prediction could surpass human intelligence3 claims
and 24 more
Most important claims · 10 of 76Search all claims for “Compute” →
- Optical fiber providers (Corning and Light) represent the next evolution in data center networking because copper wires become inefficient and generate excessive heat when used to connect very large numbers of GPUs, making optical fiber the necessary next-generation solution.Leopold Aschenbrenner says "No More Stocks!"
- Progress in AI comes from three sources—compute, data, and algorithms—all of which have been progressing at smooth exponential rates across the industry, and technological progress toward AI is not driven by any one company but is a project of all of humanity combining GPUs, algorithmic ideas, and data processing systems.Sam Altman and Greg Brockman on AI and the Future (Full Audio)
- Scaling matters profoundly for AI capabilities; there are predictable scaling laws showing smooth exponential relationships between compute/data and capability across many different axes of variation, representing a deep scientific discovery about the nature of intelligence.Sam Altman and Greg Brockman on AI and the Future (Full Audio)
- Once AI research is fully automated with millions of parallel agents running at high serial speed, the quantity of researchers and serial speed cease to be bottlenecks (with diminishing returns past ~90x serial speed); the binding constraints become research taste (ability to learn from experiments and pick the right ones) and compute for running experiments.AI 2027: month-by-month model of intelligence explosion — Scott Alexander & Daniel Kokotajlo
- OpenAI was right to leave robotics because at the time there was no path to data: progress comes from the combination of compute and data, and even a giant operation of a hundred robots could not generate enough data, whereas today a path exists if one commits to building hundreds of thousands of robots in a gradual loop of more robots, more usefulness, more data.Ilya Sutskever (OpenAI Chief Scientist) — Why next-token prediction could surpass human intelligence
- Hayek's critique of the 'man of system' is strong because a central planner is one man with limited compute who cannot process or use all the data; but AI changes this not via one super-planner but via many AIs each optimizing their local corner, processing vastly more details and connections than a Hayekian citizen can and potentially interacting with each other in Hayekian ways far faster than humans, arriving at better local solutions.Zvi Mowshowitz on AI and the Dial of Progress
- When securing compute, most cloud providers, chip fabs, and energy suppliers said no, calling the buildout impossible and reckless, but as in early-stage fundraising only one or two yeses were needed to start building; Microsoft was the first yes, followed by Oracle on cloud and Nvidia as a tremendous partner.Sam Altman - How to Make an Abundant Future - [Invest Like the Best, EP.484]
- In 2021, AI researchers concluded that solving middle and early high school competition math problems would require new science because AI could only score 6% of human-level performance, but by early 2024 these systems score above 94% of human-level performance on the same benchmark using only scale without breakthrough science.AI Today and Tomorrow
- Broad scaling of AI depends on three critical components: talent, compute and computational power (vulnerable to supply chain disruptions), and data and data scale.Alexandr Wang - A Primer on AI - [Invest Like the Best, EP. 272]
- The US currently has a decisive AI lead thanks to Nvidia chips and compute, but over the next five years China has advantages in power buildout and deployment, which will matter more as AI demands grow.The US vs China in the 21st Century