# Probability with Limited Data

I have a theoretical question which has been bugging me, involving prediction based on limited data. Say you go to the horse track and see that horse A has just won its race with horse B. The two horses are scheduled to race again. If you have to bet and the other person will pick their side of the bet (aka you want fair odds), what odds do you give horse A?

What I’ve considered so far:

1. Past win ratio – gives you 1/1=100% odds, which is absurd.
2. 50/50 – Assumes that the info about horse A winning means nothing, which doesn’t feel right.
3. “Add 1 win to each horse” -> 2/3 or 2 to 1 odds – Doesn’t feel crazy, but seems arbitrary. Why is adding a number to each the right answer and why add 1 instead of 2 or 0.5?
4. There is no answer, can’t know – If we imagine that a million different people did this for a million different races each, one would come out ahead. One method would eventually prove more effective on average for estimating the odds. It really feels like there must be a best answer.

1. Past win ratio – gives you 1/1=100% odds, which is absurd.

You are talking about Bernoulli distribution, which is a special case of binomial distribution with $n=1$. The "past win ratio" that you mention is in fact a maximum likelihood estimator of binomial parameter $p$.

1. 50/50 – Assumes that the info about horse A winning means nothing, which doesn’t feel right.

This is very arbitrary choice that ignores the data, so it would be hard to justify it.

1. “Add 1 win to each horse” -> 2/3 or 2 to 1 odds – Doesn’t feel crazy, but seems arbitrary. Why is adding a number to each the right answer and why add 1 instead of 2 or 0.5?

Adding 1 is not that arbitrary as it may sound. In fact, this estimate is a result of Bayesian beta-binomial model. If we assume binomial distribution for your variable of interest $X$ and a beta prior for the parameter $p$, then the Bayesian beta-binomial model is

$$X|p \sim \mathcal{Bin}(n, p) \\ p \sim \mathcal{Beta}(\alpha, \beta)$$

and since beta distribution is a conjugate prior for binomial, we have a clear closed-form solution for the posterior

$$p|X \sim \mathcal{Beta}(\alpha + k, \beta + n - k)$$

where $n$ is the total number of races and $k$ is a number of successes by the first horse. Choosing $\alpha = \beta = 1$ leads to using uniform prior for $p$, this is one of the "uninformative" priors. So this is equivalent to saying that before seeing your data you assumed that the probability for the first horse to win is anything from 0 to 1 with equal chance. Other commonly used "uninformative" priors are $\alpha = \beta = 1/2$, $\alpha = \beta = 1/3$, or $\alpha = \beta = \varepsilon$ where $\varepsilon$ is some very small value like $0.001$.

Of course adding 1 win for each horse means that you do not want to assume a priori that any one of them can be better. If you had some out-of-data knowledge about the horses that told you that one of them has some kind of advantage, you could use informative prior, i.e. the one that brings some assumptions to the model. When choosing appropriate values for th parameters you could recall that expected value of beta distribution is $\tfrac{\alpha}{\alpha+\beta}$, and the variance is $\tfrac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}$, and use this information for choosing appropriate values of $\alpha,\beta$.

1. There is no answer, can’t know – If we imagine that a million different people did this for a million different races each, one would come out ahead. One method would eventually prove more effective on average for estimating the odds. It really feels like there must be a best answer.

Honestly, with only a single observed race you have very little information. So your "best guess" for the probability of winning would have more to do with out-of-data knowledge that has lead to choice of procedure for making the guess, then with the data itself. So the single person out of the million, that you are talking about, will either know something about the horses that others don't, or is just lucky. If it was a single person out of million, I'd rather assume that the person was lucky.