In convergence in probability or a.s. convergence w.r.t which measure is the probability?

Question

I was presenting proofs of WLLN and a version of SLLN (assuming bounded 4th central moment) when somebody asked which measure is the probability with respect too and I realised that, on reflection, I wasn't quite sure.

It seems that it is straightforward, since in both laws we have a sequence of $X_{i}$'s, independent RVs with identical mean and finite variance. There is only one random variable in sight, namely the $X_{i}$, so the probability must be w.r.t the distribution of the $X_{i}$, right? But then that doesn't seem quite right for the strong law since the typical proof technique is then to define a new RV $S_{n} := \sum_{i=1}^{n} X_{i}$ and work with that, and the limit is inside the probability:

$ Pr \left[\lim_{n \rightarrow \infty}\frac{1}{n}\sum_{i=1}^{n} X_{i} = E[X_{i}]\right]=1 $

So now it looks as if the RV is the sums over $n$ terms, so the probability is over the distribution of the sums $S_{n}$, where $n$ is no longer fixed. Is that correct? If it is, how would we go about constructing a suitable probability measure on the sequences of partial sums?

Happy to receive intuitive responses as to what is going on as well as formal ones using e.g. real or complex analysis, undergrad probability/statistics, basic measure theory. I've read Convergence in probability vs. almost sure convergence and associated links, but find no help there.

Answer 1

The probability measure is the same in both cases, but the question of interest is different between the two. In both cases we have a (countably) infinite sequence of random variables defined on a the single probability space $(\Omega,\mathscr{F},P)$. We take $\Omega$, $\mathscr{F}$ and $P$ to be the infinite products in each case (care is needed, here, that we are talking about only probability measures because we can run into troubles otherwise).

For the SLLN, what we care about is the probability (or measure) of the set of all $\omega = (\omega_{1},\omega_{2},\ldots)$ where the scaled partial sums DO NOT converge. This set has measure zero (w.r.t. $P$), says the SLLN.

For the WLLN, what we care about is the behavior of the sequence of projection measures $\left(P_{n}\right)_{n=1}^{\infty}$, where for each $n$, $P_{n}$ is the projection of $P$ onto the finite measureable space $\Omega_{n} = \prod_{i=1}^{n} \Omega_{i}$. The WLLN says that the (projected) probability of the cylinders (that is, events involving $X_{1},\ldots,X_{n}$), on which the scaled partial sums do not converge, goes to zero in the limit as $n$ goes to infinity.

In the WLLN we are calculating probabilities which appear removed from the infinite product space, but it never actually went away - it was there all along. All we were doing was projecting onto the subspace from 1 to $n$ and then taking the limit afterward. That such a thing is possible, that it is possible to construct a probability measure on an infinite product space such that the projections for each $n$ match what we think they should, and do what they're supposed to do, is one of the consequences of Kolmogorov's Extension Theorem .

If you'd like to read more, I've found the most detailed discussion of subtle points like these in "Probability and Measure Theory" by Ash, Doleans-Dade. There are a couple others, but Ash/D-D is my favorite.