token cost
The per-query computational cost of running a large language model, billed by the amount of text (tokens) processed.
As defined in Conversations with Tyler 2023 Retrospective
Other definitions
The per-unit charge for processing text in AI models, which scales with usage.
Dara Khosrowshahi - Uber's Bet on AVs, AI, and Building a Super-App - [Invest Like the Best, EP.476]
Where it comes up · 3
- Conversations with Tyler - Dave Baszucki4 claims
- Conversations with Tyler 2023 Retrospective1 claim
- What Bonds, Oil & Gold Are Telling Us | Michael Lebowitz1 claim
Most important claims · 6 of 6Search all claims for “token cost” →
- As AI token costs fall, Roblox creators could run simulated-player tests of huge scale, such as a million hours of testing in minutes, which would enable faster iteration and higher-quality games.Conversations with Tyler - Dave Baszucki
- Within less than two years, and technologically possibly already now, an LLM trained on a specific reader could excise the parts of a book that reader already knows, enabling much faster, more efficient reading; the constraint is token cost, which will become affordable within a few years.Conversations with Tyler 2023 Retrospective
- Chamath Palihapitiya, a major AI champion, reported that his company's token costs are doubling every 45 days while productivity gains are only around 5%, forcing him to rationalize token usage, and he expects other firms will have to do the same, which could shift the industry toward open-source models.What Bonds, Oil & Gold Are Telling Us | Michael Lebowitz
- Companies spend anywhere from about $1 to $50 on AI tokens for every $100 spent on people, and Baszucki argues Roblox aims to do for gaming what AI coding tools are doing for software.Conversations with Tyler - Dave Baszucki
- Games contain code, 3D objects, avatars, NPCs, scenery, missions, and quests, making them more complex than ordinary software, so AI generation of game content could consume far more tokens than coding does.Conversations with Tyler - Dave Baszucki
- Companies differ widely in how they split spending between people and tokens, with some spending roughly $1 on tokens per $100 of people costs and others spending $50 on tokens per $100 of people costs.Conversations with Tyler - Dave Baszucki