22
$\begingroup$

The central limit theorem states that the mean of i.i.d. variables, as $N$ goes to infinity, becomes normally distributed.

This raises two questions:

  1. Can we deduce from this the law of large numbers? If the law of large numbers says that the mean of a sample of a random variable's values equals the true mean $\mu$ as $N$ goes to infinity, then it seems even stronger to say that (as the central limit says) that the value becomes $\mathcal N(\mu, \sigma)$ where $\sigma$ is the standard deviation. Is it fair then to say that central limit implies the law of large numbers?
  2. Does the central limit theorem apply to linear combination of variables?
Improve this question
$\endgroup$
3
  • 5
    $\begingroup$ Your assertion that "central limit theorem states that the mean of i.i.d. variables, as $N$ goes to infinity, becomes normally distributed" is incorrect. See my answer to this recent question which raises similar issues. Another answer to that question was posted but deleted soon thereafter, and the discussion following that answer, now also gone, discussed these issues too. $\endgroup$
    – Dilip Sarwate
    Feb 10, 2012 at 1:44
  • 1
    $\begingroup$ Why is the sample mean converging to the population mean $\mu$ a weaker result than the sample mean converging to a sample from a $\mathcal N(\mu, \sigma)$ distribution? $\endgroup$
    – Dilip Sarwate
    Feb 10, 2012 at 2:07
  • $\begingroup$ @DilipSarwate Thanks for the flag, but your comment is IMO enough reveal misconceptions in the question and reasonable answers did appear. $\endgroup$
    – user88
    Feb 10, 2012 at 10:12

3 Answers 3

Reset to default
14
$\begingroup$

The OP says

The central limit theorem states that the mean of i.i.d. variables, as N goes to infinity, becomes normally distributed.

I will take this to mean that it is the OP's belief that for i.i.d. random variables $X_i$ with mean $\mu$ and standard deviation $\sigma$, the cumulative distribution function $F_{Z_n}(a)$ of $$Z_n = \frac{1}{n} \sum_{i=1}^n X_i$$ converges to the cumulative distribution function of $\mathcal N(\mu,\sigma)$, a normal random variable with mean $\mu$ and standard deviation $\sigma$. Or, the OP believes minor re-arrangements of this formula, e.g. the distribution of $Z_n - \mu$ converges to the distribution of $\mathcal N(0,\sigma)$, or the distribution of $(Z_n - \mu)/\sigma$ converges to the distribution of $\mathcal N(0,1)$, the standard normal random variable. Note as an example that these statements imply that $$P\{|Z_n - \mu| > \sigma\} = 1 - F_{Z_n}(\mu + \sigma) + F_{Z_n}((\mu + \sigma)^-) \to 1-\Phi(1)+\Phi(-1) \approx 0.32$$ as $n \to \infty$.

The OP goes on to say

This raises two questions:

  1. Can we deduce from this the law of large numbers? If the law of large numbers says that the mean of a sample of a random variable's values equals the true mean μ as N goes to infinity, then it seems even stronger to say that (as the central limit says) that the value becomes N(μ,σ) where σ is the standard deviation.

The weak law of large numbers says that for i.i.d. random variables $X_i$ with finite mean $\mu$, given any $\epsilon > 0$, $$P\{|Z_n - \mu| > \epsilon\} \to 0 ~~ \text{as}~ n \to \infty.$$ Note that it is not necessary to assume that the standard deviation is finite.

So, to answer the OP's question,

  • The central limit theorem as stated by the OP does not imply the weak law of large numbers. As $n \to \infty$, the OP's version of the central limit theorem says that $P\{|Z_n-\mu| > \sigma\} \to 0.317\cdots$ while the weak law says that $P\{|Z_n-\mu| > \sigma\} \to 0$

  • From a correct statement of the central limit theorem, one can at best deduce only a restricted form of the weak law of large numbers applying to random variables with finite mean and standard deviation. But the weak law of large numbers also holds for random variables such as Pareto random variables with finite means but infinite standard deviation.

  • I do not understand why saying that the sample mean converges to a normal random variable with nonzero standard deviation is a stronger statement than saying that the sample mean converges to the population mean, which is a constant (or a random variable with zero standard deviation if you like).

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ I wonder what the person who downvoted my answer found objectionable or incorrect in what I said. $\endgroup$
    – Dilip Sarwate
    Dec 22, 2013 at 14:06
  • $\begingroup$ I wonder what the two persons who downvoted my answer found objectionable or incorrect in what I said. $\endgroup$
    – Dilip Sarwate
    Jul 27 at 18:18
8
$\begingroup$

For law of large numbers, you need to have all variables to be defined on the same probability space (as the law of large numbers is a statement about probability of an event determined by $\bar X_n$, for all $n$ simultaneously). For convergence in distribution, you can have different probability spaces, and that simplifies many aspects of the proofs (e.g., increasing nested spaces, very common for various triangular array proofs). But it also means you cannot make any statements concerning the joint distributions of $\bar X_n$ and $\bar X_{n+1}$, say. So no, convergence in distribution does not imply the law of large numbers, unless you have a common probability space for all variables.

Improve this answer
$\endgroup$
4
  • $\begingroup$ (+1) What you say is true, and a very important point. The triangular array allows for the variables in each "row" to live on different probability spaces than previous rows. On the other hand, if we say a priori that we are considering a sequence of iid random variables, then, implicitly they must exist on a common underlying space for the notion of independence to make much sense. $\endgroup$
    – cardinal
    Feb 10, 2012 at 3:52
  • $\begingroup$ @cardinal: so if I understand correctly, in the "simple" case where all are defined in the same space, it is the case that centrality implies the law of large numbers? or no? $\endgroup$
    – user9097
    Feb 10, 2012 at 5:38
  • $\begingroup$ @user9097 Since we are now getting into realms of fine details, which law of large numbers is being asked about? The weak law or the strong law? $\endgroup$
    – Dilip Sarwate
    Feb 10, 2012 at 10:43
  • $\begingroup$ That point is true only for the strong law of large numbers, not for the weak law $\endgroup$
    – kjetil b halvorsen ♦
    May 12, 2017 at 13:27
4
$\begingroup$

First, though there are many definitions, one of the standard forms of the central limit theorem says that $\sqrt{n}(\bar{X}_n-EX)$ converges in distribution to $\mathcal N(0, Var(X))$, where $\bar{X}$ is the sample mean of $n$ iid copies of some random variable $X$.

Secondly, suppose we have two independent random variables $X$ and $Y$. Then $$\sqrt{n}(\frac{1}{n}\sum_{j=1}^n(aX_j+Y_j) - E(aX+Y)) \to \mathcal N(0, Var(aX+Y))$$ or $$\sqrt{n}a(\bar{X}_n- EX)+\sqrt{n}(\bar{Y}_n -EY) \to \mathcal N(0, a^2Var(X)+Var(Y)).$$

In other words, a linear combination of random variables wont converge to a linear combination of normals under the CLT, just one normal. This makes sense because a linear combination of random variables is just a different random variable that CLT can be applied to directly.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ This is a good start to an answer. Here are some comments: A linear combination of (joint) normals is normal, soo, I'm not sure what your comment in that regard was intended to mean. At any rate, I suspect the OP was not thinking about linear combinations of the form you consider. Observing that $\bar X_n = \sum_{i=1}^n w_{ni} X_i$ where $w_{ni} = 1/n$ for each $i = 1,\ldots,n$, a natural question one might ask is what happens when we replace these "uniform" weights with some other (more arbitrary) ones. When do we still get a CLT? Lindeberg's CLT can be used to get at this question. $\endgroup$
    – cardinal
    Feb 10, 2012 at 3:46
  • $\begingroup$ I think with strict conditions my result will still say something about $\sum^n_{j=1}w_{nj}X_j$. Lets first define these conditions and then consider how to weaken them. Lets take $w_{nj} = w_{j}/n$ and $w_j$ to be a single, infinite sequence of non-negative reals. If the number of distinct $w_{j}$ is finite and each appears infinitely often in the sequence, my result should hold as each $w_jX$ defines a random variable and this fits into the 'linear combination' framework I gave above. Then a good question would be if we could allow the number of distinct $w$ scale with $n$. $\endgroup$
    – Daniel Johnson
    Feb 10, 2012 at 14:56
  • 1
    $\begingroup$ This is a good comment, and a nice idea, however I believe it would need some modification to work. Assume wlog that $\mathbb E X = 0$. Construct your $w_j$ as follows. Let $w_1 = 1$, $w_2 = 0$. Now, define $w_j$ inductively as follows: Set $w_j = 0$ until $\sum_{i=1}^j w_i /j \leq 1/4$. Then append ones until $\sum_{i=1}^j w_i /j \geq 1/2$. Append zeros again, then ones. Repeat ad infinitum. Now, $0$ and $1$ both occur an infinite number of times, but the variance of the rescaled mean oscillates between $1/2$ and $1/4$ (roughly). So, your stated sequence cannot converge in distribution. $\endgroup$
    – cardinal
    Feb 10, 2012 at 16:03
  • $\begingroup$ (Note: There is nothing special about the choice of $0$ and $1$, here. Also, strictly speaking the procedure you describe in the comment does not really fit within the linear-combination framework of your answer.) $\endgroup$
    – cardinal
    Feb 10, 2012 at 16:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.