Scale (in AI)
The magnitude of computational resources (compute power, data size, model parameters) applied to training AI systems, which has emerged as a primary driver of capability improvements
As defined in Facing Superintelligence (with Ben Goertzel)
Where it comes up · 1
Most important claims · 6 of 6Search all claims for “Scale (in AI)” →
- The scale of compute and data has been the main factor enabling the LLM revolution, and this same underlying factor—not the specific algorithm—will be the primary driver enabling AGI to emergeFacing Superintelligence (with Ben Goertzel)
- Goertzel doesn't think the direction of training larger and larger LLMs is 'intellectually bankrupt' because LLMs have more data than humans but aren't as intelligent, suggesting scale alone doesn't suffice for human-level AGIFacing Superintelligence (with Ben Goertzel)
- A high percentage of historical AI paradigms—including genetic algorithms, logic-based AI, and hypervectors—would work effectively if scaled up with modern compute and data, but were abandoned due to perceived fundamental limitations that were actually resource constraintsFacing Superintelligence (with Ben Goertzel)
- Logic-based AI was 'inappropriately tarred and feathered' due to its association with handcoded knowledge, but the core mechanism of using logical inference as the central engine of AGI is not fundamentally tied to this limitation and can work at scale with modern approaches to convert natural language and sensory data to logic expressionsFacing Superintelligence (with Ben Goertzel)
- We should expect the first AGI to be built using simple, easy methods available given current materials and knowledge rather than complex combinations, but historically, the first approach to solving problems is often a 'kluge' that later proves to have much simpler underlying eleganceFacing Superintelligence (with Ben Goertzel)
- Hypervectors for modeling episodic memory were significant research in the 1980s-90s but abandoned, yet the last five years have seen a huge resurgence of hypervector-based memory systems and hypervector chips, showing the paradigm works at scaleFacing Superintelligence (with Ben Goertzel)