recurrent neural network
A neural network that processes sequences by maintaining a state passed from one step to the next, used for sequence data before transformers.
As defined in 280 | François Chollet on Deep Learning and the Meaning of Intelligence
Where it comes up · 4
- 336 | Anil Ananthaswamy on the Mathematics of Neural Nets and AI2 claims
- The coming AI revolution and the question no one can ignore | Max Tegmark | Strange Loop1 claim
- Machine Intelligence and Risk Management1 claim
- 280 | François Chollet on Deep Learning and the Meaning of Intelligence1 claim
Most important claims · 5 of 5Search all claims for “recurrent neural network” →
- John Hopfield's networks, inspired by the Ising model of ferromagnets and spin glasses, are fully connected recurrent networks where connection strengths are set so a stored memory sits at an energy minimum; perturbing the network drives it to a higher energy state, after which neurons flip like magnetic moments and the system relaxes back down the energy landscape to the stored memory, enabling associative memory retrieval.336 | Anil Ananthaswamy on the Mathematics of Neural Nets and AI
- Recurrent neural networks process data as ordered sequences, and because order equates to context, building up context word by word yields much richer representations than mapping a whole sequence in one shot.Machine Intelligence and Risk Management
- The human brain is a recurrent neural network where information flows around in loops, while Transformers have no loops, and one detailed theory of what causes consciousness (subjective experiences of colors, sounds, love, etc.) is that you need these loops for it.The coming AI revolution and the question no one can ignore | Max Tegmark | Strange Loop
- Hopfield networks are fully connected recurrent neural networks where neuron outputs feed back into the rest of the network, which differs fundamentally from today's feedforward networks in which computation proceeds from input to output without feedback.336 | Anil Ananthaswamy on the Mathematics of Neural Nets and AI
- The transformer architecture, developed in late 2016 and made public in 2017, replaced recurrent networks like LSTMs (which date to the early 1990s) for sequence processing and works so well that nearly anything—images, video—can be treated as sequence data, enabling the rise of internet-scale generative AI.280 | François Chollet on Deep Learning and the Meaning of Intelligence