Elo rating
System for ranking competitive entities based on head-to-head comparisons, where a 100-point difference indicates the stronger entity will win 64% of matches, used for evaluating chatbots
As defined in Jeff Dean (Google): Exciting Trends in Machine Learning
Other definitions
A numerical system for ranking chess player strength based on competition results, creating an objective measure of skill that updates continuously
Tyler Cowen: Prepare for the Brave New World | Knowledge Project Podcast (Audio)
Where it comes up · 3
- AutoResearch explained..1 claim
- Tyler Cowen: Prepare for the Brave New World | Knowledge Project Podcast (Audio)1 claim
- Jeff Dean (Google): Exciting Trends in Machine Learning1 claim
Most important claims · 3 of 3Search all claims for “Elo rating” →
- Chess skill is measured precisely by ELO rating, which creates an objective pecking order you wake up to every day, forcing you to choose between discouragement and re-energization.Tyler Cowen: Prepare for the Brave New World | Knowledge Project Podcast (Audio)
- Gemini Pro achieved the second-highest Elo rating on the LMSYS chatbot arena (which evaluates models via user-submitted prompts and anonymous pairwise comparisons across ~30 models), demonstrating competitive performance against other state-of-the-art chatbots.Jeff Dean (Google): Exciting Trends in Machine Learning
- A chess engine built using AutoRegression improved from an initial Elo rating of 750 to 2600 Elo after AutoRegression ran multiple experiments over hours, demonstrating a 1850-point improvement.AutoResearch explained..