factual

Cat neuron from unsupervised YouTube training

An early Google Brain unsupervised model trained on 10 million randomly selected YouTube frames across 2,000 computers using 16,000 cores spontaneously developed a high-level neuron that activated for cat faces—never having been told what a cat was—and the same model, about 50x bigger than prior models, improved the state of the art on the ImageNet 20,000-category challenge by 60% relative, demonstrating that scaling neural nets works.

factualpending

Speaker

Jeff Dean

Evidence Quote

“That neural net was probably 50x bigger than one that had been trained previously, and it got good results. So that sort of said to me, "Hey, actually scaling up neural nets seems like... a good idea"”

Source

Jeff Dean & Noam Shazeer — 25 years at Google: from PageRank to AGI— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:30:41 AM

Your notes

Loading notes...