token consumption
The percentage or amount of the available context window that is actually used when processing an input; LLMs are most effective at approximately 20% token consumption and degrade steeply above 60%.
As defined in A Quick Lesson Of AI Languages
Other definitions
A pricing and usage model in which customers pay according to the number of model tokens processed, which scales with usage rather than headcount.
Building Investing Superintelligence
Where it comes up · 2
Most important claims · 8 of 8Search all claims for “token consumption” →
- Every business must go through two pricing revolutions, first to usage-based pricing and then to outcome-based pricing, and Gabe wants to skip straight to outcome pricing, charging for good investment ideas, perfect LP reports, or CIMs, because outcomes are easier to value than tokens.Building Investing Superintelligence
- Recent claims about massive context windows (e.g., 1 million tokens for Claude, 2 million for Grok) are misleading because LLMs are most effective at only approximately 20% token consumption; at 60% consumption, performance already shows steep declines.A Quick Lesson Of AI Languages
- The most cutting-edge use is Moltbook-style swarms where a PM could have 10,000 agents debate ideas for 24 hours and return one idea, which is viable because investors will pay about $50,000 for one good idea whereas few domains justify that many tokens for a single insight.Building Investing Superintelligence
- Long-horizon agentic systems (e.g., OpenClaw) require the ability to work continuously over hours while staying within the 20% effective token consumption range, which is a fundamental constraint that larger context windows do not solve.A Quick Lesson Of AI Languages
- While Tune and token-optimization techniques can reduce token consumption, they do not improve accuracy beyond markdown—demonstrating that reducing token count does not necessarily translate to better LLM performance.A Quick Lesson Of AI Languages
- Token-consumption AI businesses such as Cursor, Factory, and Claude Code can sell into parts of enterprise that already buy usage-based tools, so they can go much further commercially with fewer people than per-seat enterprise software, which requires a human-heavy sales organization.Building Investing Superintelligence
- JSON and other structured formats consume more tokens than plain text because they include repeated declarations of labels and schemas that serve as machine-parsing guidance but represent wasted token consumption for the LLM.A Quick Lesson Of AI Languages
- Rogo prices per seat because its buyers are accustomed to seat pricing and compare it to Bloomberg, FactSet, Capital IQ, and PitchBook, but seat-based enterprise sales require a human-intensive go-to-market motion that cannot scale usage 100-fold per deal.Building Investing Superintelligence