definition
Two Measures of Real-Time AI Latency
Latency is measured as the delay before the first token returns after a user submits a prompt, and the gap between subsequent tokens, with the first-token delay being much longer.
definitionpending
Speaker
GavinEvidence Quote
“We measure it in 2 ways: first, the number of milliseconds between when you submit your response and when you get your first token back”
Source
Gavin Uberti - Real-Time AI & The Future of AI Hardware - [Invest Like the Best, EP.356]— Invest Like the BestCreated: 11/17/2026, 12:35:53 PM
Your notes
Loading notes...