What is the probability space that the CLT is really being applied to?

Question

Can someone please walk me through (or cite a reference to) exactly which fixed probability space is used in an application of the CLT, especially in the finite probability space case?

My question comes from the following: generally, the CLT is stated as "let $X_i$ be an infinite sequence of IID RVs on a probability space $\Omega$ with finite mean and variance ...". I use "probability space" to mean a triplet of a set $\Omega$ of "outcomes", a $\sigma$-algebra of "events" on $\Omega$, and a probability measure defined on the $\sigma$-algebra.

Traditionally, a text might say something like "the $X_i$ represent a sample from a common distribution $F$" - i.e., $P(X_i<=a) = F(a)$ for each $i$ for some distribution $F:\mathbb R\to [0,1]\subset\mathbb R$.

In the case of an infinite $\Omega$, one often thinks of $\omega\in\Omega$ as an infinite sequence $\omega=(\omega_i)$ perhaps where each $\omega_i$ is a coin flip or a repeated measurement that might carry error. In that case, one often uses something like $X_i(\omega) = \omega_i$ in a "ticket" formulation of an RV (i.e., the RV "reads off" the $i$th entry on the "ticket" which represents $\omega\in\Omega$).

In the case of finite $\Omega$, it is generally said that the finite population $\mathcal P$ of individuals (say, people whose heights we are measuring or parcels whose weights we are estimating etc) is distinct from $\Omega$ and "leads to" probability spaces $\Omega_n$ in which the points $\omega\in\Omega_n$ represent possible samples of $n$ members of the population $\mathcal P$. But then, the CLT is intended to apply to a single, fixed $\Omega$ rather than a family of $\Omega_n$. So, it seems that things get slippery at this point when we are taking limits and start considering more than one $\Omega$.

I've seen in various places various assurances like "$\Omega$ doesn't really matter, just focus on the random variables and distributions that you have direct control over". Occasionally, there will be vague reference to Kolmogorov extension or Ionescu theorems, but those seem to be a lot more about general stochastic processes than rock bottom CLT applications.

I'm looking for a clear, step-by-step, unambiguous application of the $\Omega$ concept in this case so I can have seen it once to anyone's satisfaction and then not have to worry later that I'm missing something or misapplying the theorem.

PS: please note that I've posted a related but not at all identical, separate question which also stems from my amateur's attempt to understand this important theorem:
What is the rigorous justification for applying LLN or CLT to finite probability spaces?

Answer 1

There is a universal construction that permits us to ignore these details.

Recall that a probability space is a triple $(\Omega,\mathfrak F, \mathbb P)$ of a sample space $\Omega,$ a sigma-algebra $\mathfrak F,$ and a probability measure $\mathbb P.$

An infinite sequence in $\Omega$ is a function $\omega:\mathbb N \to \Omega,$ where conventionally the value of $\omega$ at $n\in\mathbb N$ is written $\omega_n.$ Let's call the set of such functions $\Omega^\infty.$ The axioms of set theory guarantee $\Omega^\infty$ exists.

Let $\mathfrak F^\infty$ be generated by all sets of the form

$$\mathcal E_0 \times\mathcal E_1 \times\cdots \times\mathcal E_n \times\Omega \times\Omega \cdots =\{\omega \in\Omega^\infty \mid \omega_0\in\mathcal E_0, \ldots, \omega_n\in \mathcal E_n\}.$$

Again, its existence is guaranteed axiomatically.

It is straightforward (involving only basic definitions and no analysis) to show this is a sigma-algebra on $\Omega^\infty.$ Finally, define $\mathbb P^\infty$ to be the measure determined by

$$\mathbb P^\infty\left(\mathcal E_0 \times\mathcal E_1 \times\cdots \times\mathcal E_n \times\Omega \times\Omega \cdots\right) = \mathbb P(\mathcal E_0)\mathbb P(\mathcal E_1)\cdots\mathbb P(\mathcal E_n).$$

Again, it's a matter of chasing definitions to show this is a probability measure.

Finally, whenever have any infinite sequence of random variables $X_0,X_1,X_2,\ldots$ defined on $\Omega,$ for each $n\in\mathbb N$ define a real-valued function $\mathbf X_n$ on $\Omega^\infty$ via

$$\mathbf X_n(\omega) = X_n(\omega_n)$$

and use the sequence $\mathbf X_n$ instead. The foregoing construction assures that any finite collection of these $\mathbf X_n$ are independent.

From such a sequence we may construct the partial sums given by

$$S_n(\omega) = X_0(\omega) + \cdots + X_n(\omega) = X_0(\omega_0) + \cdots + X_n(\omega_n).$$

This is also (clearly) a sequence of random variables defined on $\Omega^\infty.$ They induce the corresponding sequence of distributions,

$$F_n(x) = \Pr(S_n\le x),$$

each of which is a distribution function defined on $\mathbb R$ with values in the interval $[0,1].$

The CLT (and certain laws of large numbers) concern the limiting behavior of this sequence of distribution functions.

---

Example

Let $\Omega$ be the set of six possible positions of a die with distinguishable faces, where $\mathfrak F$ is the set of all subsets of $\Omega$ and $\mathbb P$ is the uniform probability. Let $X$ be the random variable assigning the value $1$ to one of the faces, $2$ to another, and so on. Thus, $\Pr(X = k)=1/6$ for $k\in\{1,2,3,4,5,6\}.$

A sequence of iid versions of $X,$ when applied to any sequence $\omega$ of die rolls, is the sequence of these values $(X_0, X_1, X_2, \ldots).$ The set of all sequences sharing a given finite prefix $(\omega_0, \omega_1,\ldots,\omega_n)$ is measurable and has probability $6^{-n-1}.$ Consequently, the chance that the first $n+1$ values associated with the first $n+1$ die rolls is a specific finite sequence $(X(\omega_0), X(\omega_1), \ldots, X(\omega_n))$ is also $6^{-1-n}.$