7
$\begingroup$

Can you tell me if my understanding of the CLT is correct? Maybe it's just a matter of notation.

The classical CLT states:

Let $X_1,...,X_i,...,X_n$ be a sequence of iid random variables drawn from a distribution with mean $\mu$ and variance $\sigma^2 < \infty$.

Then, let the sample avarage be equal to $\bar{X}_n = \frac{X_1+...+X_i+...+X_n}{n}$.

The CLT says, when $n$ becomes large then $\sqrt{n}(\bar{X}_n-\mu)$ is approx. normal with mean $0$ and variance $\sigma^{2}$.

What this theorem is saying is: take a random sample from a population, then compute the sample average and keep it aside, then you keep sampling let's say for additional 100,000 times, and you get additional (possibly different, at least most of them) 100,000 sample averages. Then, if you look at the sampling distribution of all these sample averages, it would be approximately normal.

Now, the generic $X_i$ is the sample average obtained from the $i$ random sampling or is it simply a random observation obtained from the $i$ random sampling? I mean, is it $\bar{X}_i$ or a single observation? What I'm asking is: should I can consider $X_i$ either as a single observation (in this case each draw means to draw a single value at each trial, and the number of random draws is $n$) or as a sample mean (and in this case I'm drawing samples fixed in size, and in this case, $n$ is the number of random samples drawn)?

Shall the same interpretation for $X_i$ hold when we present the law of large numbers?

Improve this question
$\endgroup$
4
  • $\begingroup$ you may find stats.stackexchange.com/a/599253/56940 helpful $\endgroup$
    – utobi
    Commented Jan 15, 2023 at 12:56
  • $\begingroup$ Are you saying that I can consider $X_i$ either as a single observation (in this case each draw means to draw a single value at each trial, and the number of random draws is $n$) or as a sample mean (and in this case I'm drawing samples fixed in size, and in this case $n$ is the number of random samples drawn)? $\endgroup$
    – John M.
    Commented Jan 15, 2023 at 13:05
  • $\begingroup$ $\bar X_n$ is the average of $n$ iid random variables, so itself is a random variable with a distribution. $\sqrt{n}(\bar X_n-\mu)$ is also a random variable with a distribution. The CLT is saying that as $n$ increases, the distribution of $\sqrt{n}(\bar X_n-\mu)$ converges to the distribution of $N(0,\sigma^2)$ $\endgroup$
    – Henry
    Commented Jan 15, 2023 at 17:46
  • $\begingroup$ That's clear, but what I'm questioning is: if I increase $n$, say, for instance, by one unit, what am I doing? Am I drawing a new random variable, that is, a new observation and so I'm incrasing the size of my sample (and $n$ is simply the size of the sample, and $X_i$ is random observation) or, I'm drawing a new sample, I compute a new sample mean, which adds to the $n$ already computed sample means (and $n$ refers to the number of sample averages obtained from $n$ samples drawn at random, and $X_i$ is the sample average computed from the $i$ sample)? $\endgroup$
    – John M.
    Commented Jan 15, 2023 at 18:23

3 Answers 3

Reset to default
3
+50
$\begingroup$

You already answered it yourself

$$\bar{X}_n = \frac{X_1+...+X_i+...+X_n}{n}$$

  • $\bar{X}_n$ is a sample mean.

    The subscript here refers to the sample size.

  • $X_i$ is a variable in the sample.

    The subscript here refers to the id of the variable. The id's are the integers from 1 untill n.

An example is the image below of a Galton board (from Wikipedia Matemateca (IME/USP)/Rodrigo Tetsuo Argenton)

Galton board

The distribution of the beads at the bottom is the distribution of an average (or sum) $\bar{X}$, where it relates to the sum of the movements of a bead through 5 duplicate layers of pins. Every double layer the beads will hit the pins and go straight with 50% chance, left with 25% chance, or right with 25% chance. You can see each double layer as a variable $x_i$ with possible values of $-1,0,1$ (relating to the movement of the bead). The bins at the end are the result of the sum of those $x_1+x_2+x_3+x_4+x_5$ and you can end up somewhere between $-5$ and $5$ (actually this board also has bins -6 and +6, it is not such a perfect process with only steps of -1, 0 or +1).

So you have

  • $X_i$ the value of the movement of a bead in a particular (double) layer.
  • $\bar{X}_5$ the sum of all the movements in the 5 double layers.
  • The hundreds of beads in the bottom, which are different realisations of averaging/summing a sample of bead movements. The distribution of those beads will approximate a normal distribution when we increase the size of the Galton board.
Improve this answer
$\endgroup$
0
5
$\begingroup$

So I think you have a common misunderstanding. each $X_i$ is a single observation.

Just as 1 throw of a dice has uniform distribution on 1-6 ( you don't need to consider throwing 1000s of dice), a single sample mean is approximately distributed normally ( with the approximation improving as the sample size increases).

So I can calculate probabilities on my single sample mean, just as I can calculate that eg the probability of getting a number of 4 and above with a single dice throw is 1/2. and I know that after many dice throws the proportion of times I get 4 and above should approach 1/2. But this long run effect is a consequence of saying that the probability of throwing 4-6 is 1/2.

So with one sample mean and variance, I can calculate the approximate distribution of my single sample mean, and discuss the probability that my single sample mean was greater than a given value etc.

Improve this answer
$\endgroup$
5
$\begingroup$

As extensively already discussed by others and in the comments, the Central Limit Theorem tells us about the behaviour of the distribution of standardized sample averages $\bar X_i$ as $i$ becomes large, where $X_i$ and its space play no role.

In the Central Limit Theorem, the basic underlying hypothesis is

For each $n$, let $X_1, \ldots X_n$ be independent random variables.

As $n$ changes, the underlying probability space may change, but this is of no consequence since a convergence in distribution statement is a statement about convergence of a sequence of real-valued functions on $\mathbb{R}$.

Indeed, If $X_1,\ldots,X_n$ are independent with distribution functions $F_1,\ldots,F_n$ and $$ T_n = (S_n - E(S_n))/\text{var}(S_n), $$

with $S_n = X_1,\ldots,X_n$, the distribution function of $T_n$ is completely determined by the $F_i$, and the validity of a statement about convergence in distribution of $T_n$ is also determined by $F_i$, regardless of the construction of the underlying probability space.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ As $n$ changes, the underlying probability space may change, I can see what you meant. However, it should be noted that since $X_1, \ldots, X_n, \ldots$ need to be added, they need to be defined on a common sample space. $\endgroup$
    – Zhanxiong
    Commented Feb 10, 2023 at 21:19
  • 1
    $\begingroup$ Another way to see this is that the "independence" can only be defined on a common probability space. $\endgroup$
    – Zhanxiong
    Commented Feb 10, 2023 at 21:27
  • $\begingroup$ @Zhanxiong thanks for the remark. Clearly, $X_1,\ldots,X_n$ must be defined on a common triple, but the point is that for $n +1$, we could switch to a different triple. $\endgroup$
    – utobi
    Commented Feb 10, 2023 at 21:46
  • 1
    $\begingroup$ OK, thanks for the clarification. Probably you are talking about the triangular array, which is the most general :) That is good. $\endgroup$
    – Zhanxiong
    Commented Feb 10, 2023 at 21:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.