causal

Spark Born from Netflix Prize Iterative ML Bottleneck

Spark originated at UC Berkeley when the machine learning group competing in the Netflix recommendation contest found Hadoop too slow for their highly iterative algorithms, which reload and rewrite data each pass, so the systems group proposed keeping the data in memory and iterating there.

causalpending

Speaker

Ali Ghodsi

Evidence Quote

“So in some sense, it was a tool created to be able to do really fast machine learning to win the Netflix competition.”

Source

Ali Ghodsi – The Past, Present, and Future of Big Data – [Founder’s Field Guide, EP.18]— Invest Like the Best
Created: 11/14/2026, 6:55:55 AM

Your notes

Loading notes...