Steering Subsystem
Byrnes' term for the innately-programmed subcortical part of the brain (hypothalamus, brainstem, amygdala, etc.) that contains hardwired reward/cost functions and primitive sensors.
As defined in Adam Marblestone – AI is missing something fundamental about the brain
Where it comes up · 1
Most important claims · 6 of 6Search all claims for “Steering Subsystem” →
- Channeling Byrnes, the minimum set of Steering Subsystem components needed to produce something intelligent is far smaller than the set needed for human-like social instincts and ethics; therefore it is probably possible to build powerful model-based RL optimizers (potential paperclip maximizers) lacking most human reward functions, which is why understanding the Steering Subsystem matters chiefly for alignment.Adam Marblestone – AI is missing something fundamental about the brain
- Steve Byrnes proposes a solution to Ilya Sutskever's question of how evolution encodes high-level desires: a Steering Subsystem with innate reward functions and primitive sensors (e.g. superior colliculus detecting faces/threats) trains predictors ('Thought Assessors') in the cortex/amygdala that learn which abstract world-model neurons predict innate responses, thereby robustly wiring learned features to innate rewards.Adam Marblestone – AI is missing something fundamental about the brain
- Single-cell atlas work (Fei Chen, Evan Macosko, BRAIN Initiative) shows that subcortical Steering Subsystem regions like the hypothalamus contain far more diverse and bespoke cell types than the cortex, consistent with the idea that the cortex needs only enough cell types for a repeating learning architecture while the Steering Subsystem needs many species-specific cells to implement innate reward circuits via genetic wiring.Adam Marblestone – AI is missing something fundamental about the brain
- The cortical predictor of innate responses can generalize because its input is the rich, abstract world model: even the word or concept 'spider' (a book about spiders, a room with spiders) can trigger the spider-flinch reward circuit, whereas the Steering Subsystem itself can only respond to the limited primitive features its own sensors (superior colliculus) can detect.Adam Marblestone – AI is missing something fundamental about the brain
- Innate reward functions cannot rely on plasticity-driven learning; they need either special cell types or special genetic wiring rules whereby cells express matching receptors and proteins so that 'when this cell contacts that cell, form a synapse', which is why bespoke cell types are needed to specify Steering Subsystem circuitry without learning.Adam Marblestone – AI is missing something fundamental about the brain
- Because digital minds can be copied while biological minds cannot, the tradeoffs differ: it makes economic sense to amortize more capabilities into copyable digital weights, whereas evolution had to re-pass everything through the genome and rebuild each network within a dynamic lifetime, so it amortized less into the Learning Subsystem and more into innate behaviors and bootstrapping cost functions.Adam Marblestone – AI is missing something fundamental about the brain