Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 2 claims· 1 episode

MATH 500

A benchmark dataset of 500 difficult mathematics problems used to evaluate large language models' mathematical reasoning and problem-solving capabilities

As defined in Model Distillation: How to Steal a Billion-Dollar AI


Where it comes up · 1
  • Model Distillation: How to Steal a Billion-Dollar AI2 claims

Most important claims · 2 of 2Search all claims for “MATH 500” →
  • DeepSeek R1 distilled Qwen 32B (32 billion parameters) scored 72.6 on AIME 2024 and 94.3 on MATH 500 according to DeepSeek's paper and confirmed on OpenRouter, while OpenAI's o1 mini scored 63.6 on AIME 2024 and 90.0 on MATH 500—the free model is winning.Model Distillation: How to Steal a Billion-Dollar AI
  • The result of distilling R1 into small open Qwen models should not be possible on paper—a 1/20th-sized student model maintaining most of the teacher's capability violates conventional scaling expectations.Model Distillation: How to Steal a Billion-Dollar AI