beam search
Decoding algorithm that tracks multiple top-scoring token sequences simultaneously rather than greedily selecting single best tokens, using branch-and-bound to find better overall sequences.
As defined in 10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT
Where it comes up · 2
- Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]2 claims
- 10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT1 claim
Most important claims · 3 of 3Search all claims for “beam search” →
- Language models may move toward tree search, generating many candidate continuations, evaluating them, and backtracking to the best path, as chess engines do, though this has not caught on yet.Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]
- Current beam search in language models is primitive, typically using at most four beams with shallow lookahead, and it requires roughly 4x more compute, which is a large overhead, so it has not caught on.Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]
- Beam search is a decoding mechanism that looks back multiple time steps into the future rather than just one time step, keeping track of a fixed number of top-scoring tokens at each step and expanding them in a branch-and-bound fashion until the end of the sequence, producing better sentences than greedy single-token prediction.10 years of NLP history explained in 50 concepts | From Word2Vec, RNNs to GPT