Last year I started running a little rugby score prediction competition within my family. Basically for every televised game we predict the winning team and margin of victory. You score points the closer you are to the correct margin - exact = 5pts, within 2 = 4pts, within 4 = 3pts, within 6 = 2pts, within 8 = 1pt.
I want to have an additional metric to serve as a tie break if necessary, but otherwise just a curiosity (to see how it correlates with points). Last year I simply took the arithmetic mean of the differentials. So if Player 1 is off by 2, 10 and 6 for the first 3 games then their score would be 6. This seemed okay at first as it is fair to say Player 1 is 6 away on average in that example.
A problem arises when shock results happen. Last season for example a team that was expected to win lost by 53 points. The best prediction was 65 points off. Games like this completely ruin the arithmetic mean metric. And it isn't just one-off games. The free-scoring nature of rugby means that variance is very high. If a game isn't close then scores can run away massively. I can illustrate that best with a histogram of all of our predictions across 104 games last season:
50% of our predictions were within 12 points of the correct margin. However the arithmetic means produced scores of 15.0, 15.5, 16.0, 16.4 and 17.0. Clearly the extreme scores make the arithmetic mean an unsuitable summary statistic. But what would be better? The median is an obvious but uninspiring choice. I guess I'm hoping for something more like the mean that assigns less weight to extreme values. Does anyone have any suggestions?
