chinchilla scaling law
DeepMind's finding on the compute-optimal ratio of model parameters to training data, guiding how to allocate compute between model size and data.
As defined in AGI is still 30 years away — Ege Erdil & Tamay Besiroglu
Where it comes up · 1
Most important claims · 1 of 1Search all claims for “chinchilla scaling law” →
- Empirical estimates of the returns to research effort in software (across linear/SAT solvers, computer vision, RL, language modeling) are ambiguous about whether scaling cognitive effort produces accelerating growth or merely exponential growth; combined with the fact that software progress tracks hardware progress (~30%/year matching Moore's law, with the deep-learning acceleration coinciding with compute scaling), this argues against a hyperbolic faster-than-exponential software-only singularity.AGI is still 30 years away — Ege Erdil & Tamay Besiroglu