7
$\begingroup$

Can someone please walk me through (or cite a reference to) exactly which fixed probability space is used in an application of the CLT, especially in the finite probability space case?

My question comes from the following: generally, the CLT is stated as "let $X_i$ be an infinite sequence of IID RVs on a probability space $\Omega$ with finite mean and variance ...". I use "probability space" to mean a triplet of a set $\Omega$ of "outcomes", a $\sigma$-algebra of "events" on $\Omega$, and a probability measure defined on the $\sigma$-algebra.

Traditionally, a text might say something like "the $X_i$ represent a sample from a common distribution $F$" - i.e., $P(X_i<=a) = F(a)$ for each $i$ for some distribution $F:\mathbb R\to [0,1]\subset\mathbb R$.

In the case of an infinite $\Omega$, one often thinks of $\omega\in\Omega$ as an infinite sequence $\omega=(\omega_i)$ perhaps where each $\omega_i$ is a coin flip or a repeated measurement that might carry error. In that case, one often uses something like $X_i(\omega) = \omega_i$ in a "ticket" formulation of an RV (i.e., the RV "reads off" the $i$th entry on the "ticket" which represents $\omega\in\Omega$).

In the case of finite $\Omega$, it is generally said that the finite population $\mathcal P$ of individuals (say, people whose heights we are measuring or parcels whose weights we are estimating etc) is distinct from $\Omega$ and "leads to" probability spaces $\Omega_n$ in which the points $\omega\in\Omega_n$ represent possible samples of $n$ members of the population $\mathcal P$. But then, the CLT is intended to apply to a single, fixed $\Omega$ rather than a family of $\Omega_n$. So, it seems that things get slippery at this point when we are taking limits and start considering more than one $\Omega$.

I've seen in various places various assurances like "$\Omega$ doesn't really matter, just focus on the random variables and distributions that you have direct control over". Occasionally, there will be vague reference to Kolmogorov extension or Ionescu theorems, but those seem to be a lot more about general stochastic processes than rock bottom CLT applications.

I'm looking for a clear, step-by-step, unambiguous application of the $\Omega$ concept in this case so I can have seen it once to anyone's satisfaction and then not have to worry later that I'm missing something or misapplying the theorem.

PS: please note that I've posted a related but not at all identical, separate question which also stems from my amateur's attempt to understand this important theorem:
What is the rigorous justification for applying LLN or CLT to finite probability spaces?

$\endgroup$
3
  • $\begingroup$ I don't understand the question. The central limit theorem, taken formally, says something like "for all $\Omega$, for all sequences $(X_n)_n$ of blahblahblah, we have blahblah". It doesn't speak at all about any specific $\Omega$. $\endgroup$
    – Plop
    Commented Jul 21, 2023 at 12:43
  • $\begingroup$ @Plop The issue is that for finite sample spaces $\Omega,$ such sequences don't exist. Consider the simplest non-trivial case where $\Omega = \{T,H\},$ modeling the flip of a coin. There are only four binary random variables one can define on this space and they are not independent. Where will you find an infinite sequence of such random variables to apply the CLT? $\endgroup$
    – whuber
    Commented Jul 21, 2023 at 13:43
  • $\begingroup$ It's like saying that there is an issue with "for all real number $x$ such that $x^2 = -1$, I'm Superman" because there is no real number whose square is $-1$ to apply it... There's no issue at all. $\endgroup$
    – Plop
    Commented Jul 21, 2023 at 13:49

1 Answer 1

9
$\begingroup$

There is a universal construction that permits us to ignore these details.

Recall that a probability space is a triple $(\Omega,\mathfrak F, \mathbb P)$ of a sample space $\Omega,$ a sigma-algebra $\mathfrak F,$ and a probability measure $\mathbb P.$

An infinite sequence in $\Omega$ is a function $\omega:\mathbb N \to \Omega,$ where conventionally the value of $\omega$ at $n\in\mathbb N$ is written $\omega_n.$ Let's call the set of such functions $\Omega^\infty.$ The axioms of set theory guarantee $\Omega^\infty$ exists.

Let $\mathfrak F^\infty$ be generated by all sets of the form

$$\mathcal E_0 \times\mathcal E_1 \times\cdots \times\mathcal E_n \times\Omega \times\Omega \cdots =\{\omega \in\Omega^\infty \mid \omega_0\in\mathcal E_0, \ldots, \omega_n\in \mathcal E_n\}.$$

Again, its existence is guaranteed axiomatically.

It is straightforward (involving only basic definitions and no analysis) to show this is a sigma-algebra on $\Omega^\infty.$ Finally, define $\mathbb P^\infty$ to be the measure determined by

$$\mathbb P^\infty\left(\mathcal E_0 \times\mathcal E_1 \times\cdots \times\mathcal E_n \times\Omega \times\Omega \cdots\right) = \mathbb P(\mathcal E_0)\mathbb P(\mathcal E_1)\cdots\mathbb P(\mathcal E_n).$$

Again, it's a matter of chasing definitions to show this is a probability measure.

Finally, whenever have any infinite sequence of random variables $X_0,X_1,X_2,\ldots$ defined on $\Omega,$ for each $n\in\mathbb N$ define a real-valued function $\mathbf X_n$ on $\Omega^\infty$ via

$$\mathbf X_n(\omega) = X_n(\omega_n)$$

and use the sequence $\mathbf X_n$ instead. The foregoing construction assures that any finite collection of these $\mathbf X_n$ are independent.

From such a sequence we may construct the partial sums given by

$$S_n(\omega) = X_0(\omega) + \cdots + X_n(\omega) = X_0(\omega_0) + \cdots + X_n(\omega_n).$$

This is also (clearly) a sequence of random variables defined on $\Omega^\infty.$ They induce the corresponding sequence of distributions,

$$F_n(x) = \Pr(S_n\le x),$$

each of which is a distribution function defined on $\mathbb R$ with values in the interval $[0,1].$

The CLT (and certain laws of large numbers) concern the limiting behavior of this sequence of distribution functions.


Example

Let $\Omega$ be the set of six possible positions of a die with distinguishable faces, where $\mathfrak F$ is the set of all subsets of $\Omega$ and $\mathbb P$ is the uniform probability. Let $X$ be the random variable assigning the value $1$ to one of the faces, $2$ to another, and so on. Thus, $\Pr(X = k)=1/6$ for $k\in\{1,2,3,4,5,6\}.$

A sequence of iid versions of $X,$ when applied to any sequence $\omega$ of die rolls, is the sequence of these values $(X_0, X_1, X_2, \ldots).$ The set of all sequences sharing a given finite prefix $(\omega_0, \omega_1,\ldots,\omega_n)$ is measurable and has probability $6^{-n-1}.$ Consequently, the chance that the first $n+1$ values associated with the first $n+1$ die rolls is a specific finite sequence $(X(\omega_0), X(\omega_1), \ldots, X(\omega_n))$ is also $6^{-1-n}.$

$\endgroup$
1
  • $\begingroup$ Thanks @whuber! I was familiar with this construction right up until where you say "The foregoing construction assures that any finite collection of these 𝐗𝑛 are independent." I assume you mean that the "lifted" $X_n$ are independent even if the original $X_n$ defined on $\Omega$ are not. I need to think about this a bit to convince myself... Also where you talk about the $F_n$ I would have thought the (vanilla) CLT expects those $F_n$ to be the same in order to be applicable. I assume you're saying the CLT applies if the "lifted" $X_n$ do, indeed, have identical distribution. $\endgroup$
    – ac1501
    Commented Jul 20, 2023 at 21:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.