factual
Beam Search Is Primitive and Compute-Heavy
Current beam search in language models is primitive, typically using at most four beams with shallow lookahead, and it requires roughly 4x more compute, which is a large overhead, so it has not caught on.
factualpending
Speaker
GavinEvidence Quote
“beam search also requires 4x more compute, which is a very large overhead to pay.”
Source
Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]— Invest Like the BestCreated: 11/17/2026, 12:35:53 PM
Your notes
Loading notes...