2
$\begingroup$

The the law of large numbers wikipedia page has an example on fair, six-sided dice:

According to the law of large numbers, if a large number of fair six-sided dice are rolled, the average of their values (sometimes called the sample mean) is likely to be close to 3.5 (the expected value of the average of the rolls), with the precision increasing as more dice are rolled.

What confuses me is the mention of "sample mean" in above text. Wikipedia defines it as:

The sample mean is the average value (or mean value) of a sample of numbers taken from a larger population of numbers, ...

In this specific example (repeated rolling dice experiments), I wonder what's the population and what's the sample?

My guess is that the population here are an infinite number of experiment results (in theory) and a sample is an actual experiment result (in practice). Is this understanding correct? I'm not sure about it because

  1. In my understanding, population usually means a collection of objects which have a common feature to be measured. But in this case, the population seems to be a collection of measured results (that is, data points) instead. Or maybe we should consider the infinite number of experiments as the population and the feature to be measured is experiment's output?

  2. Also, my guess that a sample is _an_ actual experiment result seems to conflict with the statement in wikipedia that "the average of their values is sometimes called the sample mean", which implies _all_ actual experiment results is a sample.

Or could it be that, when used with random variable, the "sample" in sample mean has a quite different meaning than that in the usual "sample vs population" discussion?

Thank in advance if anyone can share how you think about it.

$\endgroup$

2 Answers 2

2
$\begingroup$

We can model a "large number" $n$ of dice by listing all possible sequences of outcomes on slips of paper. "Fair" means every possible sequence occurs equally often. Thus, observing a sequence of $n$ rolls is (in terms of probabilities) exactly the same as picking one of these $6^n$ slips of paper randomly and uniformly.

For instance, $n=8$ rolls would be modeled with slips on which the sequences 11111111, 11111112, 11111113, ..., and on through 66666665 and 66666666 have been written.

A statistic is a definite mathematical operation performed on each slip of paper. The "sample mean" is a statistic computed by summing all the values on a slip and dividing by their count. Continuing the example, the sample means (which may also be written on the same slips of paper) would be $1,$ $9/8,$ $10/8,\ldots$ and on through $47/8$ and $48/8.$

The tickets with the sample means written on them are the "population" of sample means.

When we roll the dice $n$ times, the "sample" is indicated by withdrawing single ticket with a sequence written on it. This models a single experiment consisting of the $n$ rolls.

The fundamental idea is that a complex experiment can be recorded on a single slip of paper and modeled, probabilistically, as if the experimenter merely had to withdraw that slip from a box containing all possible results (in proportions reflecting the probabilities). This powerful conceptual simplification underpins most of modern statistical thinking.


To illustrate, here is a bar chart displaying the frequencies of all $6^8$ sample means in the example.

enter image description here

Here is a bar chart displaying the frequencies of all $6^{80}$ sample means for a sequence of $n=80$ rolls of the die.

enter image description here

An here is the chart for $6^{800}$ rolls:

enter image description here

(It has 4001 vertical bars but most are too short to render.)

The quoted law of large numbers says that as $n$ increases, this chart squeezes horizontally into an arbitrarily narrow curve. (Specifically, given any number $1-\epsilon \lt 1$ and any interval $[a,b]$ spanning the value $3.5,$ there is an $N$ for which the proportion of the sample means for rolling $n\ge N$ dice falling in the interval $[a,b]$ exceeds $1-\epsilon.$) Visually, when $n$ is sufficiently large these plots focus on $3.5,$ which is the expectation of a single die.

$\endgroup$
3
  • $\begingroup$ Thanks for your explanation. I'm a statistics newbie (I just read about it occasionally). I can follow your explanation, but I wonder if it's better/simpler to model it (a larger number of repeated experiments) in a different way as follows? 1) There are infinite dices (this is the population). All of them are same. 2) Pick n dices (this is the size of the sample) and perform the experiment with each of them. 3) the average of the experiment results are sample mean. $\endgroup$
    – rayx
    Commented Aug 30, 2023 at 1:57
  • $\begingroup$ I see you emphasized the fundamental idea in your approach. I agree it's very helpful, though I'm not sure if it's a widely used approach (I didn't read about it elsewhere, perhaps because I'm a newbie?). Also, as mentioned in my answer, maybe it's best to avoid terms like sample or population in the first place when discussing repeated experiments? I upvoted your answer but keep the question open for now. Thanks. $\endgroup$
    – rayx
    Commented Aug 30, 2023 at 2:11
  • 1
    $\begingroup$ For a reference to this approach and terminology, see any edition of Freedman, Pisani, & Purves, Statistics. As soon as you postulate an infinity of something you must be extremely careful and mathematically formal (and even then you risk confusion and paradox). IMHO it is far better to stick to finite sets when you don't require an actual infinity. $\endgroup$
    – whuber
    Commented Aug 30, 2023 at 13:36
0
$\begingroup$

Below is my own attempt to answer the question.

TL;DL There are two versions of the law of large numbers. One for repeated experiments, another for experiments on different members in a population.

The law of large numbers can be proved by using Chevyshev's inequality, which is a property of a random variable (note that there is no population, sample, etc. involved to understand Chevyshev's inequality). What law of large numbers says is that, if we repeat an experiment multiple times, the average of their results converges to the expected value of the experiment (note again there is no population, sample, etc. involved in this statement).

Now let's interpret that statement in different contexts:

  • In the context of repeating an experiment on a single object (i.e., dice) which generates different results, the average of their results is (X1+X2+...+Xn)/n, where Xi is the nth experiment.

  • In the context of measuring a feature of the members in a sample from a population, the average of the measures is sample mean.

So, as far as the law of large numbers is concerned, "the average of repeated experiment results on a single object" and "sample mean" are same thing in different context. And there is no need to introduce population and sample concepts in the former context.

That seems quite a reasonable explanation to me :)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.