Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 2 claims· 2 episodes

Chinchilla

A DeepMind result showing optimal training requires scaling data (tokens) proportionally with model parameters, not just making models bigger.

As defined in Aravind Srinivas


Where it comes up · 2
  • Aravind Srinivas1 claim
  • Cursor Team1 claim

Most important claims · 2 of 2Search all claims for “Chinchilla” →
  • OpenAI's GPT insight was that the transformer made language modeling roughly 100x more efficient at reaching the same performance than RNNs, so running the same compute yields a far better model; combined with scaling laws and the Chinchilla insight that you must scale both parameters and tokens (and token quality/dedup), this evolution — not attention alone — produced the breakthrough.Aravind Srinivas
  • The original OpenAI scaling laws paper was slightly wrong due to issues with learning rate schedules, and Chinchilla later showed a more correct compute-optimal version, after which practitioners deviated again to optimize for inference budget rather than pure compute-optimality.Cursor Team