AlexNet
The 2012 breakthrough neural network, trained on a couple of cheap NVIDIA GPUs via CUDA, that suddenly let computers identify images with unprecedented accuracy, launching the modern AI era.
As defined in The Man Who Built NVIDIA (with Stephen Witt)
Where it comes up · 9
- No Priors Ep. 39 | With OpenAI Co-Founder & Chief Scientist Ilya Sutskever4 claims
- Yann LeCun's $1B Bet Against LLMs [Part 1]4 claims
- Ilya Sutskever – We're moving from the age of scaling to the age of research1 claim
- AGI is still 30 years away — Ege Erdil & Tamay Besiroglu1 claim
- Ilya Sutskever – We're moving from the age of scaling to the age of research1 claim
- The Man Who Built NVIDIA (with Stephen Witt)1 claim
- Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton1 claim
- Jeff Dean (Google): Exciting Trends in Machine Learning1 claim
and 1 more
Most important claims · 10 of 15Search all claims for “AlexNet” →
- Around 2012, fringe researchers used two ~$1,000 NVIDIA GPUs and CUDA to jerry-rig a supercomputer that simulated brain neurons, producing AlexNet—suddenly computers could identify images with unprecedented accuracy on cheap commodity hardware, proving the missing ingredient for neural nets was firepower and inaugurating NVIDIA's AI phase.The Man Who Built NVIDIA (with Stephen Witt)
- LeCun pioneered convolutional neural networks in the 1980s during expert systems era, and 25 years later AlexNet's breakthrough architecture was uncannily similar to LeCun's 1990s convolutional nets, validating his early insights.Yann LeCun's $1B Bet Against LLMs [Part 1]
- In 2012, Alex Krizhevsky and Ilya Sutskever developed AlexNet, a neural network that dramatically outperformed existing computer vision systems at object recognition in images, which opened the floodgates for neural networks to take over the AI field.Will AI outsmart human intelligence? - with 'Godfather of AI' Geoffrey Hinton
- Children learn general representations with very few explicitly labeled examples, unlike supervised learning approaches like AlexNet that required the enormous manually-labeled ImageNet dataset, suggesting human learning relies on different mechanisms than supervised learning.Yann LeCun's $1B Bet Against LLMs [Part 1]
- The ability to train large neural networks at AlexNet scale required not just the existence of gradient descent as an algorithm, but specific technical insights about how to make gradient descent work in practice, which most machine learning researchers did not possess at the time.No Priors Ep. 39 | With OpenAI Co-Founder & Chief Scientist Ilya Sutskever
- Pre-AlexNet, neural networks were marginalized in AI research because researchers could not prove mathematical theorems about them, and the prevailing academic culture valued theoretical provability as a criterion for good research.No Priors Ep. 39 | With OpenAI Co-Founder & Chief Scientist Ilya Sutskever
- AlexNet was trained on 2 GPUs, the original Transformer paper used 8-64 GPUs maximum, and o1 reasoning was not the most compute-heavy thing, demonstrating that research breakthroughs do not require the absolute maximum available compute, only sufficient compute to validate ideas.Ilya Sutskever – We're moving from the age of scaling to the age of research
- Over the past decade AI gained major capability 'unlocks' (game play, language, abstract reasoning/coding/math) at a rate of roughly one every three years or one per three orders of magnitude of compute scaling; since AGI needs several more such unlocks (long-horizon coherence, agency, full multimodality) and only ~3-4 orders of magnitude of compute scaling remain before data centers consume a non-trivial fraction of world output, full automation should take decades, not years.AGI is still 30 years away — Ege Erdil & Tamay Besiroglu
- In 2012, the AlexNet paper by Krizhevsky, Sutskever, and Hinton achieved a 13% improvement in ImageNet accuracy (from ~51% to ~64%), which was remarkable and a major breakthrough because they were the only entrants using neural networks out of ~28 competitors, and this led nearly all subsequent entrants to adopt neural networks.Jeff Dean (Google): Exciting Trends in Machine Learning
- However, by 2021 fully supervised approaches like Google's Vision Transformer achieved 88.6% ImageNet accuracy, showing self-supervised methods were still inferior despite Barlow Twins' breakthrough.Yann LeCun's $1B Bet Against LLMs [Part 1]