Brier Score to predict unpredictability

In the latest Betting Insiders magazine from The Betting School we wrote an article that included some testing of midweek fixtures using the Brier score. The idea for this came from the Pinnacle Betting Blog which often has interesting guest articles on it. This is how they describe the Brier score and how to use it for football

The Brier Score, as originally introduced in 1950, is a measure of the effectiveness of weather predictions.

Calculating the Brier Score

As probabilities implied by betting markets add up to more than 100%, we first adjust the odds pro-rata to determine the probabilities of each outcome for a match.

The Brier Score per match is the sum of the square difference of the probability and actual results. Let’s take the match Liverpool vs Crystal Palace on 8th November. Pinnacle Sports odds implied a 58.3% chance of Liverpool to win and a 24.5% chance of draw. The outcome was a Crystal Palace win at a predicted possibility of just 17.1%.

The probabilities for Win:Draw:Loss were 0.583:0.245:0.171 with the actual result being 0:0:1, where one stands for the actual outcome. The differences are 0.583:0.245:0.829. The sum of the square differences are 0.5832 + 0.2452 + 0.8292 = 1.0875. This is the Brier Score for this match.

The Brier score can range from 0, if odds guessed the outcome directly by stating there is 100% chance of only one outcome to be correct, to 2, if odds implied that one outcome is certain and that did not occur.

However, if we had to just throw outcomes at random and say that a home win, draw or away win are equally likely, then our Brier score would be 0.667, irrespective of the outcome. 

The table below shows the Brier score per team in the Premiership. What we note is that some of the surprises are actually vindicated in the stats: Chelsea, Liverpool and West Ham are not performing as expected while for Arsenal and Manchester City it is business as usual.

You can see on the Pinnacle website how they applied this to check on individual teams and in a future blog we’ll apply it to different months in the football season.