# Testing the validity of models for adjusting bookmaker odds into probabilities of real-world events

I'm looking at the use of bookmaker odds to predict the outcome of sporting events in which only two results are possible. A problem with using bookmaker odds to predict outcomes is that they include some vigorish, i.e. a profit margin, typically around 5%.

So for instance a bookmaker might offer decimal odds of 1.2 ($$\frac{1}{1.2} = 0.833$$ probability) for Team A to win, and decimal odds of 4.5 ($$\frac{1}{4.5} = 0.222$$ probability) for Team B to win. $$0.833 + 0.222 = 1.055$$, so here the margin is 5.5%. Since we can't know how bookmakers actually decide their margins, I had to model the margins and come up with estimated probabilities that sum to one.

I have four models. So for instance one model adjusted Team A's decimal odds from 1.2 to 1.2390 (0.8071 probability), and adjusted Team B's decimal odds from 4.5 to 5.1833 (0.1929 probability). Now $$0.8071 + 0.1929 = 1$$.

I then obtained bookmaker odds for about 1400 games, and ran a simulation. For each game in the simulation, $1 is bet on each team. Thus in that example I gave above, \$1 would be bet on Team A at odds of 1.2390, and \$1 would be bet on Team B at odds of 5.1833. If Team A won the game then \$1 × 1.2390 - \$1 = \$0.2390 would be gained, but the \$1 bet on Team B would be lost, and thus there would be an overall loss of \$0.7610. If Team B won the game then \$1 × 5.1833 - \$1 = \$4.1833 would be gained, but the \$1 bet on Team B would be lost, and thus there would be an overall profit of \$3.1833. This process was repeated for about 1400 games, and the results are as follows: Having done that simulation, I'm unsure how to interpret the results and assess the models. It seems the purple Model 4 is giving odds for the underdog team that are too short. Although Model 4 is also giving rather generously long odds for the favorites, the amount lost betting with poor odds on the underdogs causes the overall total profit to be negative. Meanwhile, it seems like the yellow Model 3 is doing the opposite. It's giving generously high odds for underdogs. Although this is counterbalanced to some extent by the fact that it gives relatively short odds for the favorites, the amount won winning big on the underdogs causes the overall total profit to be strongly positive. Model 1 and Model 2 are somewhere in between and thus seem to be doing a better job of estimating the true probabilities. 1. How can I decide which model is best? It seems Model 1 (in blue) and Model 2 (in red) are best at representing true odds since they're closest to$0 profit, but how can I work out how much data I need to properly establish this? Are there confidence intervals or things of that sort that can be applied to this data? A friend suggested I try some sort of bootstrapping approach, although it's not obvious to me how I would implement that.

2. Is my approach an appropriate way to validating the models? What would be other or better ways of doing that?

• I could be totally mistaken, but it sounds like you have not understood how book makers work: they set the odds so that the \$ amount of bets on either side offset so they make money either way ie the idea is that by setting high enough odds unpopular teams will have enough bets to counteract bets on the favourite Sep 8 '18 at 14:23
• I'm aware that bookmakers usually respond to incoming bets by adjusting the odds they offer so as to achieve the outcome you described. Are you able to explain the conflict you perceive between that and what I have written? Sep 8 '18 at 22:45
• You need to do multiple runs for each model. These are more or less random walks. Oct 11 '18 at 10:26
• Also, do you have a dataset of real-life games for which you know both the outcome and the odds? You can test your model by using the adjusted probabilities to predict the game outcome directly, using a proper scoring rule (1, 2, 3). Oct 11 '18 at 10:31
• I do have a dataset of such games, and so I could easily follow the Merkle & Steyvers paper you kindly linked and rank the models according to some of the proper scoring rules they mentioned (Brier Score, Logarithmic Score, etc), and then select the model which overall does the best. However, I was a little unsure how that related to your earlier comment about needing to do multiple runs for each model? Oct 11 '18 at 11:49

So for the bookies to be rational bookies and for the players to be rational players it follows that $$O_{stated}\le\frac{1-p_{callibrated}}{p_{callibrated}},$$ where $$O$$ is the stated odds and $$p$$ are well-calibrated probabilities.

You should do a Bayesian regression but condition the coefficient around the stated odds to be 1. You know that the stated odds should be a scaling of the true odds. So $$O_{stated}\pi=O_{callibrated},$$ where $$\pi$$ is $$(1+ \text{profit margin}),$$ where $$\pi>1$$.

If you assume that the margin is constant, then this is a non-simple logistic regression. I say it is not simple because you have two restrictions.

First, your scaling coefficient is just your regression constant, but it is bound between zero and one. Second, your coefficient against your data has to be conditioned to unity. So, if you would think of this as $$Y=\beta_1O_{stated}+\beta_0,$$ it must be the case that $$\beta_1=1$$ and $$\beta_0>0$$. Since you believe the margin is around 5%, your prior expectation should be around $$\beta_0\approx{\log(1.05)}$$, where $$Y$$ is the binary for a win or loss.

If you relaxed these restrictions, it would imply that someone is using something other than well-callibrated odds. From the research on parimutuel betting, that would be a surprise. You could, then do a model selection process to see which method produced a more probable result.

You should be using a Bayesian logistic regression.

• Thanks for the thoughtful response (+1). You mention that "If you assume that the margin is constant, then this is a non-simple logistic regression." As it happens the margins are expected to vary. Among other things, bookmakers will take bigger markets on particular events, and will also take bigger margins for games in which one side is a big underdog. How would that impact your suggested solution? Sep 10 '20 at 12:21
• @user1205901-ReinstateMonica I would add any regressors that you believe would cause a change in the spread, such as the prime lending rate. I would also model non-linearly in stated odds. I may add a threshold where if the odds are large enough, an additional margin would be created. The first thing I would do is run the above regression and see if I saw patterns in the graph I may not have accounted for. I might partition the sample by time as well. Sep 10 '20 at 13:20