router
A component (in GPT-5 or in MoE layers) that decides which model or expert handles a given query/token, used to route most users to cheaper models to save compute.
As defined in Ai Sota #2026
Other definitions
Software that sends each AI query to the best model for cost and quality, used by AI application companies.
Gavin Baker - AI, Semiconductors, and the Robotic Frontier - [Invest Like the Best, EP.385]
Where it comes up · 4
- Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI1 claim
- Alex Sacerdote - How to Invest Through Technology Cycles - [Invest Like the Best, EP.477]1 claim
- Gavin Baker - AI, Semiconductors, and the Robotic Frontier - [Invest Like the Best, EP.385]1 claim
- Ai Sota #20261 claim
Most important claims · 4 of 4Search all claims for “router” →
- Contrary to claims that it is hard to decode what each expert does, in the original Mixture of Experts work the experts were relatively easy to understand—one expert handled words referring to cylindrical objects, another was good at dates and times—though you don't need that human understanding to operate the model, since a learned router selects experts based on the example.Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI
- GPT-5's headline feature being a router likely saved OpenAI significant money because most users are no longer charging their GPU costs as much by being routed to a smaller, non-thinking model.Ai Sota #2026
- Unlike cloud servers, which are largely commodity with stickiness coming from software on top, frontier models are highly differentiated by training methods and skills, such as Anthropic's strength in private equity and finance and Google's strength in ingesting PDFs, and routers that switch between models mask this differentiation.Alex Sacerdote - How to Invest Through Technology Cycles - [Invest Like the Best, EP.477]
- Apple will monetize AI by using a router to direct queries to the best cloud model per dollar and charging AI companies increasing tolls to be included, and will offer tiered subscriptions for cloud intelligence, similar to how Google collected tolls from search.Gavin Baker - AI, Semiconductors, and the Robotic Frontier - [Invest Like the Best, EP.385]