Long context creates a massive compute inefficiency: for a 500-page manual (~250k tokens), every user query requires reprocessing the entire manual. RAG, by contrast, pays the embedding cost only once at indexing time, not on every request.

causalpending

Speaker

Unidentified Speaker — Is RAG Still Needed? Choosing the Best Approach for LLMs [UabBYexBD4k]

Evidence Quote

“For a 500-page manual, that's something like 250k tokens, and we need to process that every time we make a user query. RAG pays that processing cost only once at indexing time.”

Source

Is RAG Still Needed? Choosing the Best Approach for LLMs— IBM Technology
Created: 8/13/2026, 7:43:56 AM

Your notes

Loading notes...