Brier scores
A scoring rule measuring the accuracy of probabilistic forecasts by penalizing the squared difference between assigned probability and actual outcome over many events
As defined in Philip Tetlock on Superforecasting
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “Brier scores” →
- In the IARPA forecasting tournaments, the Good Judgment Project's teams—including motivated amateurs—generated more accurate probability estimates than rival university teams, an internal prediction market (Inkling), and classified U.S. intelligence analysts who had access to secret information.Philip Tetlock on Superforecasting
- It is impossible to measure the accuracy of a probability judgment for a single event, except in limiting cases where someone assigns probability zero or 1.0; accuracy can only be assessed across many events and time periods, where good judgment means consistently assigning higher probabilities to things that happen than to things that don't.Philip Tetlock on Superforecasting