neural net
A form of AI that simulates the firing of neurons in the human brain; considered a fringe, dead technology around 2011 but revived once enough computing power was applied.
As defined in The Man Who Built NVIDIA (with Stephen Witt)
Other definitions
A computational system inspired by biological brains, composed of interconnected nodes that perform weighted operations; the underlying architecture of large language models like ChatGPT
CHATGPT won't ever be the same again after this
Where it comes up · 16
- Facing Superintelligence (with Ben Goertzel)8 claims
- Stephen Wolfram - Where the Computational Paradigm Leads (in Physics, Tech, AI, Biology, Math, ...)6 claims
- Geoff Hinton - Will Digital Intelligence Replace Biological Intelligence? | Vector's Remarkable 20245 claims
- CHATGPT won't ever be the same again after this5 claims
- Ray Kurzweil (USA) at Ci2019 - The Future of Intelligence, Artificial and Natural4 claims
- Ilya Sutskever – We're moving from the age of scaling to the age of research3 claims
- The Man Who Built NVIDIA (with Stephen Witt)3 claims
- Stephen Wolfram tackles the big questions—from God and free will to whether AI will destroy us all.2 claims
and 8 more
Most important claims · 10 of 47Search all claims for “neural net” →
- Because multiplier circuit area scales quadratically (p×q) with bit width, halving the precision should more than quadruple — not merely double — the achievable FLOP count, which is the single biggest reason low-precision arithmetic has worked so well for neural nets; Nvidia historically doubled FLOPs per halved precision but only acknowledged the larger speedup (3x FP4 vs FP8) in B300 and beyond, though it arguably should be 4x.Dwarkesh Patel Podcast - Reiner Pope – Chip design from the bottom up
- Because of Turing's 1936 results (the halting problem), it is mathematically impossible to know in advance where a program will go or what a computation will produce; each step is deterministic but the outcome is not predetermined, so the only way to know the result is to run the computation.325 | Alvy Ray Smith on Pixar, Pixels, and the Great Digital Convergence
- Around 2011, neural nets were considered a dead, fringe technology—Russell and Norvig's ~1100-page AI textbook devoted only about 16 pages to them—and AI itself was viewed as a career graveyard with near-zero venture capital, yet AI's missing ingredient was simply computing power.The Man Who Built NVIDIA (with Stephen Witt)
- Around 2012, fringe researchers used two ~$1,000 NVIDIA GPUs and CUDA to jerry-rig a supercomputer that simulated brain neurons, producing AlexNet—suddenly computers could identify images with unprecedented accuracy on cheap commodity hardware, proving the missing ingredient for neural nets was firepower and inaugurating NVIDIA's AI phase.The Man Who Built NVIDIA (with Stephen Witt)
- The architectural similarity between McCulloch-Pitts neurons and biological neurons, preserved in modern artificial neural networks, may be fortunate and responsible for human-aligned generalization, but this alignment is not obviously necessaryCHATGPT won't ever be the same again after this
- If you train an LLM on all music up to 1900 without data after 1900, it will never invent neoclassical metal, grindcore, progressive jazz, hip-hop—it won't synthesize new genres the way humans did.Facing Superintelligence (with Ben Goertzel)
- Symbolic AI researchers criticized Hinton's 1985 language model approach as a silly way to solve language problems, noting that inductive logic programming could produce similar results by directly searching discrete rule spaces; but as scale increased, the neural approach proved far superior to rule-based manipulation.Geoff Hinton - Will Digital Intelligence Replace Biological Intelligence? | Vector's Remarkable 2024
- The perceptron, a single-layer artificial neural network introduced in the 1960s, could recognize some patterns but had fundamental limitations, such as inability to solve the connectedness problem (determining whether shapes are spatially connected), which Minsky's 1969 book mathematically proved.Ray Kurzweil (USA) at Ci2019 - The Future of Intelligence, Artificial and Natural
- In 1985, Hinton created a small language model (112 training cases initially, then ~1000) trained to predict the next word to learn representations of word meanings, which demonstrated that feature vectors for word meanings and interactions between features could explain how meaning relates to other words, reconciling the psychological feature-vector theory and structuralist relational theories of meaning.Geoff Hinton - Will Digital Intelligence Replace Biological Intelligence? | Vector's Remarkable 2024
- AI can be understood two ways: through classic economics it is just another productivity-improving tool that makes life better; through biology, new emergent entities (like oxygen-producing algae or migrating predators) often destroy what existed and remake the world in their image—so whether AI evolves into a self-perpetuating organism with its own will determines whether we get the utopian economic outcome or the dangerous biological one.The Man Who Built NVIDIA (with Stephen Witt)