8
$\begingroup$

I was presenting proofs of WLLN and a version of SLLN (assuming bounded 4th central moment) when somebody asked which measure is the probability with respect too and I realised that, on reflection, I wasn't quite sure.

It seems that it is straightforward, since in both laws we have a sequence of $X_{i}$'s, independent RVs with identical mean and finite variance. There is only one random variable in sight, namely the $X_{i}$, so the probability must be w.r.t the distribution of the $X_{i}$, right? But then that doesn't seem quite right for the strong law since the typical proof technique is then to define a new RV $S_{n} := \sum_{i=1}^{n} X_{i}$ and work with that, and the limit is inside the probability:

$ Pr \left[\lim_{n \rightarrow \infty}\frac{1}{n}\sum_{i=1}^{n} X_{i} = E[X_{i}]\right]=1 $

So now it looks as if the RV is the sums over $n$ terms, so the probability is over the distribution of the sums $S_{n}$, where $n$ is no longer fixed. Is that correct? If it is, how would we go about constructing a suitable probability measure on the sequences of partial sums?

Happy to receive intuitive responses as to what is going on as well as formal ones using e.g. real or complex analysis, undergrad probability/statistics, basic measure theory. I've read Convergence in probability vs. almost sure convergence and associated links, but find no help there.

$\endgroup$
  • 1
    $\begingroup$ You may also be interested in this recent question, which touches on some related points. $\endgroup$ – cardinal May 19 '11 at 15:12
  • $\begingroup$ @cardinal Thanks - and +1 for the x-ref which is clearly related. $\endgroup$ – Bob Durrant May 23 '11 at 9:25
12
$\begingroup$

The probability measure is the same in both cases, but the question of interest is different between the two. In both cases we have a (countably) infinite sequence of random variables defined on a the single probability space $(\Omega,\mathscr{F},P)$. We take $\Omega$, $\mathscr{F}$ and $P$ to be the infinite products in each case (care is needed, here, that we are talking about only probability measures because we can run into troubles otherwise).

For the SLLN, what we care about is the probability (or measure) of the set of all $\omega = (\omega_{1},\omega_{2},\ldots)$ where the scaled partial sums DO NOT converge. This set has measure zero (w.r.t. $P$), says the SLLN.

For the WLLN, what we care about is the behavior of the sequence of projection measures $\left(P_{n}\right)_{n=1}^{\infty}$, where for each $n$, $P_{n}$ is the projection of $P$ onto the finite measureable space $\Omega_{n} = \prod_{i=1}^{n} \Omega_{i}$. The WLLN says that the (projected) probability of the cylinders (that is, events involving $X_{1},\ldots,X_{n}$), on which the scaled partial sums do not converge, goes to zero in the limit as $n$ goes to infinity.

In the WLLN we are calculating probabilities which appear removed from the infinite product space, but it never actually went away - it was there all along. All we were doing was projecting onto the subspace from 1 to $n$ and then taking the limit afterward. That such a thing is possible, that it is possible to construct a probability measure on an infinite product space such that the projections for each $n$ match what we think they should, and do what they're supposed to do, is one of the consequences of Kolmogorov's Extension Theorem .

If you'd like to read more, I've found the most detailed discussion of subtle points like these in "Probability and Measure Theory" by Ash, Doleans-Dade. There are a couple others, but Ash/D-D is my favorite.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ +1, I've started to write my own explanation, but yours is way better. $\endgroup$ – mpiktas May 19 '11 at 14:03
  • 1
    $\begingroup$ +1. May I just add, for the visually oriented, that we often plot the sample path $(n, \frac{1}{n}S_n)$ and expect that the graph will level of around the expectation for large $n$. This is precisely what the SLLN says will happen. The WLLN says something that is more vague, and it does not guarantee that the graph will level of eventually. Since the SLLN holds, it will, but there are other counterexamples that show that we can have convergence in probability and yet with probability one the sample path will not level of. $\endgroup$ – NRH May 19 '11 at 16:21
  • $\begingroup$ @mpiktas, Thanks. @NRH, yes, you are exactly right. If I get some time later this evening maybe I can add a graph like you were talking about. $\endgroup$ – user1108 May 19 '11 at 21:24
  • $\begingroup$ @NRH Thanks for the suggestion; I made some graphs, but they seemed better placed in the other question here: stats.stackexchange.com/questions/2230/… $\endgroup$ – user1108 May 20 '11 at 2:49
  • $\begingroup$ @G. Jay, great, and you got +1 for that answer too. $\endgroup$ – NRH May 20 '11 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.