8
$\begingroup$

I might be missing something basic - but it appears that the strong law of large numbers covers the weak law. If that case, why is the weak law needed?

$\endgroup$
10
$\begingroup$

The most general case of the Weak Law of Large Numbers does not even require the existence of first moments. Therefore, it holds under conditions/assumptions more general than the conditions/assumptions required for the Strong Law of Large Numbers (existence of first moments).

Allow me to quote for you the relevant results from Durrett, Probability: Theory and Examples (4th edition), so you can see the truth of the above statement for yourself.

(p.60) Theorem 2.2.7 Weak Law of Large Numbers Let $X_1, X_2, \dots$ be i.i.d. with $$x \mathbb{P}(|X_i|>x) \to 0 \quad as \quad x \to \infty \quad (for\,all\,i=1,2,\dots) $$ Let $S_n = X_1 + \dots + X_n$ and $\mu_n = \mathbb{E}[X_1 1_{|X_1 \ge n|}]$. Then $S_n/n - \mu_n \to 0$ in probability.

The condition, for each $X_i$ in the sequence of random variables, that $x \mathbb{P}(|X_i| > x) \to 0$ as $x \to \infty$ is strictly weaker than the existence of first moments -- i.e., there exist i.i.d. sequences of random variables which satisfy this condition but which do not have finite first moments. For an example, see the previous answer above.

(p.73) Theorem 2.4.1. Strong Law of Large Numbers Let $X_1, X_2, \dots$ be pairwise independent identically distributed random variables with $\mathbb{E}|X_i| < \infty$ (for all $i = 1, 2, \dots$). Let $\mathbb{E}X_i = \mu$ and $S_n = X_1 + \dots + X_n$. Then $S_n/n \to \mu$ almost surely as $n \to \infty$.

Theorem 2.4.5. on p.75 is the Strong Law for the case that the first moment exists but is not finite.

Both results (the Weak Law of Large Numbers and the Strong Law of Large Numbers) are a lot easier to prove if/when we assume that the random variables have finite variance (second moments), but such an assumption is unnecessary for both results.

So, in conclusion, the Weak Law of Large Numbers is not redundant, because although its conclusion is weaker than that of the Strong Law of Large Numbers, it is true "more often" (i.e. under more general conditions) than the Strong Law of Large Numbers. So even when the Strong Law doesn't hold, the Weak Law may still hold.

$\endgroup$
  • 1
    $\begingroup$ we know that the condition you state for the weak law is necessary, but are the conditions that the RVs are pairwise iid and the mean exists necessary for the strong law (in the sense of the conclusion of thm 2.2.7 with "in probability" replaced by "almost surely")? $\endgroup$ – user795305 Jun 30 '17 at 17:12
  • 2
    $\begingroup$ No, pairwise iid is not, you can substitute other conditions (the Kolmogorov criterion, for example, for independent variates with different variances (therefore not iid): $\sum \sigma_k^2/k^2$ converges). There are also conditions for correlated variates. It also doesn't require a mean: ams.org/journals/tran/1973-185-00/S0002-9947-1973-0336806-5/…, but again, other conditions need to hold. $\endgroup$ – jbowman Jun 30 '17 at 17:21
  • $\begingroup$ @Ben The mean has to exist, because the mean doesn't exist for the Cauchy distribution, and the SLLN (nor the WLLN) holds for the Cauchy distribution. I don't know about pairwise independence. The proof given in Durrett is due to Etemadi (1981). In 1997 Etemadi published a paper in which he claimed that in the 1981 paper it was shown that the SLLN holds if and only if $\mathbb{E}(|X_i|) < \infty$ i.e. are necessary and sufficient. sciencedirect.com/science/article/pii/S016771529600123X Note that the "SLLN" in Durrett for mean existing but not finite isn't the same conclusion as SLLN $\endgroup$ – Chill2Macht Jun 30 '17 at 17:22
  • 1
    $\begingroup$ The version of the WLLN above is called the Feller WLLN or the Kolmogorov-Feller WLLN, see: www2.stat.duke.edu/courses/Fall09/sta205/lec/lln.pdf or: link.springer.com/article/10.1023/B:JOTP.0000040299.15416.0c The conditions for the Kolmogorov-Feller WLLN are not only sufficient but also necessary; see: stat.umn.edu/geyer/8112/notes/weaklaw.pdf I am not sure if weaker assumptions are possible for convergence in probability to infinity (i.e. for the WLLN) in case the mean exists but is infinite. $\endgroup$ – Chill2Macht Jun 30 '17 at 17:46
  • $\begingroup$ The conditions of the Feller WLLN imply $\mathbb{E}|X_1|^{1-\epsilon} < \infty$ for some $\epsilon >0$, but I am not sure if it is possible for the mean to exist and be infinite with $\mathbb{E}|X_1|^{1-\epsilon} \not< \infty$ for all $\epsilon > 0 $. So perhaps the infinite mean version of the SLLN might hold for some cases where the Feller WLLN does not; I don't know; at the very least generally though one does not have the infinite mean case in mind when talking about the SLLN, and the finite mean version of the SLLN definitely holds under less general assumptions than the Feller WLLN. $\endgroup$ – Chill2Macht Jun 30 '17 at 17:50
4
$\begingroup$

The mathematical formulations of the "Strong" and "Weak" Laws of Large Numbers look somewhat similar. Yet, the two Laws are quite different in nature :

The Weak Law never considers infinite sequences of realizations of a random variable. It only states that imbalanced sequences are less likely to occur as one considers longer sequences.

On the other hand, the Strong Law considers only infinite sequences of realizations of a random variable, and more precisely, the set of these infinite sequences. It states that the set of imbalanced sequences has probability 0 in a sense that generalizes the concept of "set of measure 0".

It can be shown that the Strong Law implies the Weak Law, which can therefore be regarded as a consequence of the Strong Law.

The converse is, however, wrong : it is possible to exhibit sequences of r.v.s following the Weak Law, but not the Strong one. So the terms "Weak" and "Strong" are indeed justified. For example, Let your sequence be i.i.d. with density

$f_X(x)=x^{-2}I(x>1)$

You can obtain a WLLN but not a SLLN, due to the Borel-Cantelli lemma.

$\endgroup$
  • 1
    $\begingroup$ Can you please clarify what you are trying to say in the final two sentences of the post (For example...)? $\endgroup$ – cardinal Jan 30 '13 at 22:48
  • 2
    $\begingroup$ I believe that $I(x >1)$ is the indicator function for the event $x > 1$, i.e. $0$ for $x \le 1$ and $1 \ge 1$. Thus the function $f_X(x) = x^{-2}I(x > 1)$ equals $0$ for $x \le 1$ and $x^{-2}$ for $x \ge 1$. All of the variables in the sequence are independent of one another, and are distributed such that they have the function $f_X$ as their probability density function. Then because of the Borel-Cantelli Lemma, this sequence satisfies the conclusion of the WLLN, but it does not satisfy the conclusion of the SLLN. $\endgroup$ – Chill2Macht Jun 30 '17 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.