AIME 2024
A standardized benchmark for mathematical reasoning ability consisting of difficult high school math competition problems, used to evaluate large language models' mathematical capabilities
As defined in Model Distillation: How to Steal a Billion-Dollar AI
Where it comes up · 1
Most important claims · 4 of 4Search all claims for “AIME 2024” →
- The 7 billion parameter version of DeepSeek R1 distilled Qwen, small enough to run on a decent laptop, still clears 55% on AIME 2024—achieving 1/20th the size of the teacher with most of the capability intact.Model Distillation: How to Steal a Billion-Dollar AI
- DeepSeek R1 distilled Qwen 32B (32 billion parameters) scored 72.6 on AIME 2024 and 94.3 on MATH 500 according to DeepSeek's paper and confirmed on OpenRouter, while OpenAI's o1 mini scored 63.6 on AIME 2024 and 90.0 on MATH 500—the free model is winning.Model Distillation: How to Steal a Billion-Dollar AI
- The gap between teacher and student is real and grows in specific ways; on AIME 2024, the full R1 teacher scores 79.8 while the 32B student scores 72.6, a 7-point gap.Model Distillation: How to Steal a Billion-Dollar AI
- The result of distilling R1 into small open Qwen models should not be possible on paper—a 1/20th-sized student model maintaining most of the teacher's capability violates conventional scaling expectations.Model Distillation: How to Steal a Billion-Dollar AI