5
$\begingroup$

I cannot understand what I am doing wrong here, so please somebody, point it out from me.

The issue? I keep finding that the Lindeberg sufficient condition for the Central Limit Theorem for independent non-identically distributed random variables, holds already by the premises (i.e. holds always given the premises). The condition is

$$\lim_{n \to \infty} \frac{1}{s_n^2}\sum_{k = 1}^{n} \mathbb{E}\big[(X_k - \mu_k)^2 \cdot \mathbf{1}_{\{ | X_k - \mu_k | > \varepsilon s_n \}} \big] = 0,\;\;\; \forall \varepsilon > 0$$

where $s_n^2 \equiv \sum_{k=1}^n \sigma_k^2$,

and I keep finding that as long as

$$0<c\leq\sigma^2_k< \infty,\;\;\; \forall\,k$$

which is part of the premises, the condition will hold. We have

$$\mathbb{E}\big[(X_k - \mu_k)^2 \cdot \mathbf{1}_{\{ | X_k - \mu_k | > \varepsilon s_n \}} \big] \\= \mathbb{E}\left[(X_k - \mu_k)^2\mid \{| X_k - \mu_k | > \varepsilon s_n\}\right] \cdot \text{Prob}\left(| X_k - \mu_k | > \varepsilon s_n\right)$$

$$ \leq \sigma^2_k \cdot \text{Prob}\left(\frac{|X_k - \mu_k|}{\sigma_k}> \varepsilon s_n/\sigma_k\right) $$

because conditional variance is not greater than unconditional variance

$$\leq\sigma^2_k \cdot \frac {\sigma^2_k}{\varepsilon^2 s_n^2}= \frac {(\sigma^2_k)^2}{\varepsilon^2 s_n^2}$$

by the basic Chebychev inequality. Then

$$\lim_{n \to \infty} \frac{1}{s_n^2}\sum_{k = 1}^{n} \mathbb{E}\big[(X_k - \mu_k)^2 \cdot \mathbf{1}_{\{ | X_k - \mu_k | > \varepsilon s_n \}} \leq \lim_{n \to \infty} \frac{1}{s_n^2}\sum_{k = 1}^{n} \frac {(\sigma^2_k)^2}{\varepsilon^2 s_n^2}$$

$$=\frac 1{\varepsilon^2 } \cdot \lim_{n \to \infty} \frac{\sum_{k = 1}^{n}(\sigma^2_k)^2}{\left(\sum_{k=1}^n \sigma_k^2\right)^2} = \frac 1{\varepsilon^2 } \cdot \lim_{n \to \infty} \frac{\sum_{k = 1}^{n}(\sigma^2_k)^2}{\sum_{k = 1}^{n}(\sigma^2_k)^2 + \sum_{k \neq j}\sigma^2_k\sigma^2_j}$$

Having all variances strictly positive and finite makes this limit zero which sandwiches the Lindeberg condition to zero. Where is the mistake here?

(Hmmm... except if what is needed for the last limit to go to zero is that all variances are not just finite, but also bounded, which is not part of the premises)

$\endgroup$
  • $\begingroup$ Not a complete answer, but glancing through: it is not necessarily true that conditional variance is less than unconditional variance. Perhaps this is where your mistake lies? $\endgroup$ – Cliff AB Dec 21 '15 at 0:01
  • 2
    $\begingroup$ By making $\sigma_n^2$ grow large sufficiently fast, you will violate the premises. For instance, $\sigma^2_n = 2^n$ will work. $\endgroup$ – whuber Dec 21 '15 at 0:10
  • 1
    $\begingroup$ @whuber Well, this example appears to validate my last-minute thought, i.e. that it is indeed not true that the Lindeberg condition holds by the premises alone, because my line of reasoning requires that we have to assume additionally that the variances are also bounded. Thanks. $\endgroup$ – Alecos Papadopoulos Dec 21 '15 at 2:27
  • 3
    $\begingroup$ Consider the following: $X_1 \sim N(0,1), X_2 = 0$ and $Z \sim$ Bernoulli$(p = 0.5$). If $Y = Z \times X_1 + (1 - Z) \times X_2$, then Var($Y$) < Var($Y | Z = 1$). $\endgroup$ – Cliff AB Dec 21 '15 at 2:43
  • 1
    $\begingroup$ btw, I think the rule you were thinking of that lead to the confusion is that $E_X[Var_Y ( Y | X) ] \leq Var(Y)$ $\endgroup$ – Cliff AB Dec 27 '15 at 21:26
3
$\begingroup$

User @CliffAB spotted the unjustified step here, which is to assume that the conditional variance is never greater than the unconditional one.

A quick simulation: $100,000$ draws from a standard normal, $Z$.
Sample standard deviation of the full sample : $0.997$.
Now, cut out the body of the distribution and keep the tails (which is in the spirit of the conditioning event in the Lindeberg condition) : specifically, keep only the draws where $|Z|>1.65$. Survived: $9,805$ draws. Standard deviation of them : $2.097$.

Intuition : when we keep both tails, it is as though we tend to transform a continuous distribution into a discrete one, creating two lumps of probability mass, and this tends to increase the variance. Consider the Bernoulli $(0.5)$ distribution and the Uniform $(0,1)$. The first is the extreme discrete version of the second, and it has three times greater variance ($1/4$ as opposed to $1/12$). And the evolution is gradual : consider a discrete uniform taking values $\{0,0.25,0.75,1\}$. Its variance is $9/64$, which is inbetween the previous two.

$\endgroup$
  • $\begingroup$ Haha, still haven't read it throughly enough to know why that's important, but glad to hear that it is. $\endgroup$ – Cliff AB Dec 21 '15 at 4:18
  • $\begingroup$ @CliffAB Because I used a "smaller than and sandwich" argument based on this wrong assumption. It is still my impression that with bounded variance terms the Lindeberg condition will hold, but I need to show it in some other way. $\endgroup$ – Alecos Papadopoulos Dec 21 '15 at 10:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.