MATH 500
A benchmark dataset of 500 difficult mathematics problems used to evaluate large language models' mathematical reasoning and problem-solving capabilities
As defined in Model Distillation: How to Steal a Billion-Dollar AI
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “MATH 500” →
- DeepSeek R1 distilled Qwen 32B (32 billion parameters) scored 72.6 on AIME 2024 and 94.3 on MATH 500 according to DeepSeek's paper and confirmed on OpenRouter, while OpenAI's o1 mini scored 63.6 on AIME 2024 and 90.0 on MATH 500—the free model is winning.Model Distillation: How to Steal a Billion-Dollar AI
- The result of distilling R1 into small open Qwen models should not be possible on paper—a 1/20th-sized student model maintaining most of the teacher's capability violates conventional scaling expectations.Model Distillation: How to Steal a Billion-Dollar AI