factual

Disk Versus Memory Processing Bottleneck

Early big data processing ran mostly on disk because memory was expensive, so each computation step loaded data from disk, processed it, and wrote it back, which was far slower than in-memory processing, while CPUs were the component that stagnated.

factualpending

Speaker

Ali Ghodsi

Evidence Quote

“So the data sets on disk, you load it in, you do some competition on it, you write it back to disk because you don't have enough memory to store all of it in memory.”

Source

Ali Ghodsi – The Past, Present, and Future of Big Data – [Founder’s Field Guide, EP.18]— Invest Like the Best
Created: 11/14/2026, 6:55:55 AM

Your notes

Loading notes...