# What's the difference between biased outcomes and independent outcomes?

Specifically, suppose you're considering outcomes on a roulette wheel. Roulette is a game of independent trials meaning that "the wheel has no memory" and past outcomes are no predictor of future outcomes (because in the case of roulette, draws are "with replacement" so each bet has the same probability of winning on each spin of the wheel).

Now I've been having an argument with someone who maintains that in roulette, if outcomes are biased (owing to some defect in the wheel, perhaps) they cannot at the same time be independent. His reasoning is that in this case past results are an indicator of future outcomes, so there is a dependence between spins in this case. i.e. by recording spin results and comparing them with the theoretical distribution, a bias can be identified, if it exists.

Furthermore, he claims that all roulette wheels are imperfect and thus biased to some degree, therefore roulette isn't a game of independent trials at all! (assuming of course that you're playing on a real wheel, not a software based random number generator).

I think there's something very fishy about this argument, but I'm not able to articulate what it is. Can anyone help?

• Be careful, because "biased" has a different meaning in statistics. It looks like you are using this word to refer to a non-uniform distribution of probabilities for the outcomes. NB: your argument is readily resolved by consulting the definition of "independent": see en.wikipedia.org/wiki/… for instance.
– whuber
Dec 4 '17 at 16:09
• It is not the future outcomes which are updated by what you observed, but what you know of the probability model which generates those future outcomes. When I realize my coin is biased towards heads by 60%, I'm not more likely to have a tails because I've now observed such an excess of heads: that would still be Gambler's fallacy. Dec 4 '17 at 16:32

Short answer: This is quite a subtle issue relating to the interaction of marginal and conditional independence. The outcomes of the Roulette wheel are conditionally independent (conditional on the underlying distribution of outcomes) but they are marginally positively correlated when you do not condition on the underlying distribution. Your friend is essentially correct, though he might not be explaining this distinction properly.

Full answer: This exact issue has been discussed in detail in a series of academic papers on binomial and multinomial prediction in Bayesian analysis (O'Neill and Puza 2005; O'Neill 2009; O'Neill 2012; O'Neill 2015). Those papers examine models where there is an exchangeable sequence of outcomes, which leads to the IID statistical model. Most importantly, the observable outcomes in the model are independent and identically distributed conditional on the underlying distribution (or equivalently, its indexing parameters), but this does not mean they are marginally independent.

As discussed in detail in O'Neill (2009), when we examine the marginal distribution of the observable values from an exchangeable sequence (i.e., not conditional on knowledge of their underlying distribution), we see that there is no longer independence between the observable outcomes, and indeed, these generally have positive correlation. Theorem 2 of that paper is reproduced here in slightly different language.

Consider an exchangeable sequence $$X_1,X_2,X_3,...$$. From the representation theorem, the values of the sequence are IID conditional on the underlying distribution $$F_X$$ (which we will take to be indexed by a parameter $$\theta$$). Define the conditional mean and variance as:

$$\mu(\theta) \equiv \mathbb{E}(X_i|\theta) \quad \quad \quad \sigma^2(\theta) \equiv \mathbb{V}(X_i|\theta).$$

For all $$i \neq j$$ we have:

\begin{aligned} \mathbb{Cov}(X_i,X_j) &= \mathbb{E}(X_i X_j) - \mathbb{E}(X_i) \mathbb{E}(X_j) \\[6pt] &= \mathbb{E}(\mathbb{E}(X_i X_j|\theta)) - \mathbb{E}(\mathbb{E}(X_i|\theta)) \mathbb{E}(\mathbb{E}(X_j|\theta)) \\[6pt] &= \mathbb{E}(\mu(\theta)^2) - \mathbb{E}(\mu(\theta))^2 \\[6pt] &= \mathbb{V}(\mu(\theta)) \geqslant 0, \\[6pt] \end{aligned}

and:

$$\mathbb{Corr}(X_i,X_j) = \frac{\mathbb{Cov}(X_i,X_j)}{\mathbb{V}(X_i)} = \frac{\mathbb{V}(\mu(\theta))}{\mathbb{V}(\mu(\theta)) + \mathbb{E}(\sigma^2(\theta))} \geqslant 0.$$

(Unless $$\mu(\theta)$$ is almost surely constant, these inequalities are strict.)

This marginal probability result reflects the fact that observed outcomes give information on the underlying distribution, which alters our beliefs in such a way that we believe that future outcomes are likely to be the same or similar to observed values. This all occurs explicitly in Bayesian analysis, but in the context of frequentist analysis it is a bit trickier, since frequentists never make probability statements that are not conditional on the underlying parameters. The cited paper discusses this issue in detail, and notes that in this context there is no explicit correlation, but there is a form of pseudo-correlation that arises under the frequentist paradigm.

If you would like to learn more about this issue, I recommend reading those papers as a starting point. Application of this situation to gambling games and the gambler's fallacy is discussed in detail. In regard to Roulette, your friend is correct that, if there is any non-zero probability of "bias" in the wheel, then ---looking at the marginal distribution of the outcomes--- the outcomes are positively correlated (and hence are used to predict each other). It is also correct that the outcomes remain conditionally independent, given knowledge of the underlying distribution (and hence the underlying "bias" of the wheel).

The issue is in this statement, which is not quite right:

past outcomes are no predictor of future outcomes

Independent means that past outcomes do not influence future outcomes.

If you flip a coin and get heads three times, that has no influence on whether the next result will be heads or tails.

However, previous independent observations certainly can help you predict what will happen in a future outcome.

If you flip a coin 10,000 times and find that you get heads about 50% of the time, you can be confident that you have a 50% chance of getting heads the next time.

If you spin a (biased) roulette wheel 10,000 times, and it comes up black 60% of the time, that helps you predict that you are more likely to get black than red the next time.

But the probability distribution of the different outcomes has no bearing on whether the observations are independent. If I manufacture a very crude coin that lands on heads 90% of the time, observations are every bit as independent as a perfect coin with an equal probability of heads and tails.

• So obviously independence and bias are not mutually exclusive. I've been trying to think of an example of dependent outcomes. What about if someone was playing a system in which he bet heavily on black after 5 reds in a row, and the casino was cheating by using magnets attached to the wheel which deflected the ball away from black and were turned on when the gambler started betting. There would then be a regular pattern to the outcomes, and the occurrence of the 5 reds WOULD affect the probability of occurrence of the next outcome. This would be a case of non-independent outcomes, right?
– Suno
Dec 4 '17 at 17:42