When we roll a fair die, why is 6 the expected number of rolls after which we get our first 3?

Question

When we roll a fair die, why is the expected number of rolls after which we get our first 3 (or any number from 1 to 6) is 6 rolls?

This code finds the expected number of rolls after which we get our first 3(or any other number from 1 to 6) when we roll our die 10000 times:

s=0
p=1/6
for i in range(1,10**4):
  s+=((i*p)*((1-p)**(i-1)))
print(s)

output:
6.000000000000001

It makes some sense that after (approximately) 6 rolls I would have gotten all the 6 numbers on a fair die, but can someone explain it in a more intuitive way and also give a mathematical proof?

Why is the expected number of rolls after which we get our first 3, in 10000 rolls, the reciprocal of the probability of getting a 3 in 1 roll?

Answer 1

This answer consists of two parts. The first part develops a basic insight about long sequences of repetitions of an experiment. This insight is conveyed by a simple diagram of the experimental results. The second part quickly answers the question by applying this insight.

The insight

Consider any probabilistic event, such as a 3 appearing in one roll of a die ("experiment $A$"). Its "probability" is intended to reflect the proportion of times this event occurs in very long sequences of the experiment.

One way to compute this proportion is to run a slightly different experiment, "experiment $B.$" The second version repeats experiment $A$ up until the moment a prescribed outcome, such as 3, appears. Let's refer to this outcome as $\omega.$ Let $N$ count how many iterations of experiment $A$ are needed until $\omega$ occurs. When we repeat experiment $B$ we observe a sequence of realizations of such random variables: $N_1$ for the first occurrence, $N_2$ for the second, and so on.

This figure shows a schematic timeline in which the repetitions of experiment $A$ are plotted left to right. Each occurrence of $\omega$ is noted. The $N_i,$ by definition, count how many trials of experiment $A$ were needed to produce each successive $\omega.$ Evidently, $\omega$ occurs $n$ times out of $N_1+\cdots + N_n$ repetitions of experiment $A.$

Because experiment $A$ (rolling a die) is assumed to behave the same way each time and to have independent outcomes, the $N_i$ have identical distributions and are independent, too. Let's use them to estimate how often $\omega$ appears in a long sequence of runs of experiment $A.$ Pick a large number $n$ of iterations of experiment $B,$ with outcomes $N_1, N_2,\ldots, N_n.$ This implies that $\omega$ occurred in exactly $n$ out of $N_1+N_2+\cdots+N_n$ iterations of experiment $A.$ The proportion estimates the chance of $\omega$ in experiment $A:$

$$\Pr(\omega) \approx \frac{n}{N_1+N_2+\cdots + N_n} = \frac{1}{\frac{1}{n}\sum_{i=1}^n N_i}.$$

(The second equality arises from the algebra of fractions: numerator and denominator were both divided by $n.$)

In the denominator appears an approximation to the expected value of experiment $B.$ As a matter of notation, let $N(\omega)$ refer to the generic outcome of experiment $B,$ so that we may express this fact as

$$E[N(\omega)] \approx \frac{1}{n}\sum_{i=1}^n N_i.$$

Weak laws of large numbers guarantee these approximations become arbitrarily good as $n$ increases. Putting the results together, we see that

$$\Pr(\omega) = \frac{1}{E[N(\omega)]}.$$

The application

A die is fair when all its outcomes are equally likely. With a six-sided die then, the sum of all six chances must be $1$ (axiomatically), implying each chance is $1/6.$ In the denominator of the foregoing result we can read off the expected time to roll any given face: it is $6,$ QED.

Answer 2

There are two forms of the geometric distribution. The one we use here counts the number $X$ of Bernoulli trials until the first Success occurs, where the Success probability is $p,$ so that the PDF is $$f_X(x) = p(1-p)^x,$$ for $x = 1, 2, 3, \dots$ and $$E(X) = \mu_X = 1/p.$$ [The alternative version counts the number of Failures before the first Success.]

It is not trivial to show that $$E(X) = \sum_{x=1}^\infty xf_X(x) =\sum_{x=1}^\infty xp(1-p)^{x-1} = 1/p.$$ [The Wikipedia article linked above shows a formal derivation for the alternative version. A slight modification works for our version.]

However, the terms of the sum decrease markedly as $x$ increases, so that one does not need to sum a huge number of terms to get a good approximation. For example, let $p = 1/6.$

p = 1/6; x = 0:100;  f = p*(1-p)^{x-1}
mu = sum(x*f)
[1] 6

In your problem about rolling a fair die, the probabilty of getting a 3 on any one roll is $p = 1/6,$ so the expected number of rolls of the die until a 3 occurs is $6.$

Notes: (1) One way to show that $\mu_X = 1/p$ is to use moment generating functions. The proof if the Wilkipedia article uses an analogous differentiation method.

(2) The geometric distribution has the memoryless property: $P(X > m+n | X > m) = P(X > m),$ for positive integers $m, n.$ So the average number of rolls after the first 5 observed until we get a 3 is also $6.$

(3) An approximate simulation of a million waiting times for the first 3 shows that the average wait is $6.$ [Extremely rare waits longer than 100 trials are ignored.]

set.seed(2021)
w = replicate(10^6, match(3, sample(1:6, 100, rep=T)))
mean(w)
[1] 6.003519

Answer 3

For a nonnegative integer-valued discrete random variable $X$, it is a standard result (see e.g. here on stats.SE) that $$E[X] = \sum_{n=0}^\infty P(X > n).$$ When $X$ is a geometric random variable with parameter $p$ that takes on values $1, 2, 3, \ldots$ (which is the case here since $X$ is the number of the trial on which $3$ occurs for the first time), we have that \begin{align} E[X] &= \sum_{n=0}^\infty P(X > n)\\ &= 1 + (1-p) + (1-p)^2 + (1-p)^3 + \cdots\\ &= \frac{1}{1 - (1-p)}\\ &= \frac 1p \end{align} without the need for simulations as in BruceET's answer or taking derivatives and worrying about interchanging the order of differentiation and summation etc as in the Wikipedia article referenced in BruceET's answer.

Intuitively, if $3$ has probability $\frac 16$ of occurring, relative frequency notions say that over a long run of $N$ trials, $3$ should occur on roughly $\frac N6$ trials, and so the average spacing between successive occurrences of $3$ should be $5$, that is, on average, every sixth trial results in a $3$.