8
$\begingroup$

Exercise 15.5.1 from Klenke's "Probability Theory: A Comprehensive Course" reads as follows. Find a sequence $X_1, X_2, \ldots$ of independent real random variables with $\mathbb{E}[|X_n|]=\infty$ for all $n\in \mathbb{N}$ such that $$ \frac{X_1+ \cdots + X_n}{\sqrt{n}} \stackrel{n \to \infty}{\Longrightarrow} N(0,1). $$ I'm not sure how this is possible if the mean isn't even defined on this case. All the cases I can think of of variables with undefined mean do not satisfy a Central Limit Theorem with a $\sqrt{n}$ scaling. Any help is appreciated.

$\endgroup$
5
  • 1
    $\begingroup$ Hint: Reduce the concepts to their basic elements. Since the Normal is at the heart of the CLT, start with iid standard Normal $X_i$. Then modify them so that (a) each has infinite expected absolute value while (b) the amount of modification decreases rapidly with $n$, so that in the limit division by $\sqrt{n}$ "kills off" the modified part. A simple way to modify a random variable is to add another one to it. $\endgroup$
    – whuber
    Dec 8, 2016 at 14:41
  • $\begingroup$ @whuber Is your point that the random variables are not said to be identically distributed? If they were i.i.d. I would agree with the OP. $\endgroup$ Dec 8, 2016 at 15:57
  • $\begingroup$ @whuber Also does the answer require that only Xn has the infinite mean. $\endgroup$ Dec 8, 2016 at 16:01
  • $\begingroup$ I guess the "for all n" means that all the Xi have infinite mean absolute value. I am trying to be careful here. I want to understand your hint but not say something that gives away the solution to the OP. $\endgroup$ Dec 8, 2016 at 16:10
  • 1
    $\begingroup$ @Michael If they were iid then the result would not be true. The idea is to make the part of $X_n$ that causes the infinite expectation to have less and less chance of occurring as $n$ increases. A good way to do that is with a mixture. $\endgroup$
    – whuber
    Dec 8, 2016 at 16:49

1 Answer 1

7
$\begingroup$

Among the many ways to solve this one, constructing the sequence by perturbing a standard Normal variable seems like the simplest and most elegant.

At the end I comment on the connection with the Central Limit Theorem.


Characteristic Functions

Allow me a digression before I present a solution. The inspiration for the technique that will be used comes from the idea that there is more than one way to describe the distribution of any random variable $X$. Commonest, and most direct, is its distribution function $F_X(x)=\Pr(X\le x)$. An indirect but extremely useful alternative is its characteristic function

$$\psi_X(t) = E\left[e^{itX}\right] = E\left[\cos(t X)\right] + i\, E\left[\sin(t X)\right].$$

Because $|e^{itX}|=1$ for all $t$, $\psi_F$ is defined for any distribution $F$ (and its values for all $t$ cannot exceed $1$ in size). Moreover, $X$ and $Y$ have the same distribution if and only if they have the same characteristic function. Even better is Lévy's Continuity Theorem: A sequence $X_n$ converges in distribution to a random variable $X$ if and only if for every $t$ the sequence $\phi_{X_n}(t)$ converges to a value $\psi(t)$ and the function $\psi$ is continuous at $0$. (All characteristic functions are continuous at $0$.) In that case, $\psi$ is the characteristic function of $X$.

Another of the lovely properties enjoyed by characteristic functions is their relationship with linear combinations: when $X$ and $Y$ are random variables (on the same probability space and $\alpha$ and $\beta$ are real numbers,

$$\psi_{\alpha X+\beta Y}(t) = \psi_X(\alpha t)\psi_Y(\beta t).\tag{1}$$

This makes characteristic functions (cfs) a suitable tool for studying perturbations of random variables $X$ achieved by adding tiny amounts of other random variables $Y$ to them: that is, random variables of the form $X+\beta Y$ for $|\beta|$ small.


Solution

Construction of a sequence

Let's construct a solution by starting with a standard Normal variable $Z$ and forming an independent sequence $Z_1, Z_2, \ldots, Z_n, \ldots$ with the same distribution as $Z$. This obviously has the limiting property we want: the means are all standard Normal, so in the limit the mean is standard Normal.

Its cf is

$$\psi_Z(t) = e^{-t^2/2}.\tag{2}$$

For the perturbations, pick some random variable $Y$ with infinite expectation. It will be convenient for $Y$ to have a cf that's easy to work with. I would like to suggest the Lévy Distribution (aka Stable Distribution with $\alpha=1/2,\ \beta=1$ or Inverse Gamma$(1/2,1/2)$ distribution) for which

$$\psi_Y(t) = e^{-\sqrt{|t|}\,(1 - i \operatorname{sgn}(t))}.$$

(For $t\gt 0$, $\operatorname{sgn}(t)=1$; for $t \lt 0,$ $\operatorname{sgn}(t)=-1$.)

This distribution is supported on $(0,\infty)$ and has no finite moments.

To this sequence of standard normal variables $(Z_n)$ let's add ever-smaller positive multiples of $Y$. (Positivity is unnecessary but it makes working with the $\operatorname{sgn}$ function easier.) Let the sequence of multiples be $p_1,p_2,p_3,\ldots,$ to be determined. Thus, the sequence of random variables is defined to be $$X_n=Z_n + p_n Y_n$$ where $(Y_n)$ is an iid sequence of random variables with the same distribution as $Y$.


Intuition

What we need to worry about is whether the perturbations are so bad that they ruin the convergence to a standard Normal distribution. To those with experience with such heavy-tailed distributions, this is a real concern: there will always be some positive probability that the little bit of $Y_n$ added into $Z_n$ will occasionally introduce such a whopping big outlier that it overwhelms the partial sum $S_n$. The entire reason for using characteristic functions is to demonstrate this will not happen in the long run, provided we reduce the amount of perturbation (the $p_n$) sufficiently rapidly.


Formal calculations

First, $X_n$ has infinite expectation because

$$E[X_n] = E[Z_n + p_n Y_n] = E[Z] + p_n E[Y] = p_n E[Y]$$

must be infinite since $E[Y]$ is infinite. Thus this sequence $(X_n)$ satisfies all the requirements of the problem.

Let's turn to the analysis of the partial means. Repeated application of $(1)$ to the partial mean

$$S_n = \frac{X_1 + X_2 + \cdots + X_n}{\sqrt{n}}$$

gives

$$\eqalign{ \psi_{S_n}(t) &= \left[e^{-(t/\sqrt{n})^2/2}\color{Blue}{\psi_Y(p_1 t/\sqrt{n})}\right] \cdots \left[e^{-(t/\sqrt{n})^2/2}\color{Blue}{\psi_Y(p_n t/\sqrt{n})}\right] \\ &= \left[e^{-(t/\sqrt{n})^2/2} \cdots e^{-(t/\sqrt{n})^2/2}\right] \left[\color{Blue}{\psi_Y(p_1t/\sqrt{n}) \cdots \psi_Y(p_nt/\sqrt{n})}\right] \\ &= e^{-t^2/(2n) - t^2/(2n) - \cdots - t^2/(2n)}\quad \color{Blue}{e^{\sqrt{|p_1t/\sqrt{n}|}(-1+i\operatorname{sgn}(p_1t/\sqrt{n})} \cdots e^{\sqrt{|p_nt/\sqrt{n}|}(-1+i\operatorname{sgn}(p_nt/\sqrt{n})} }.\tag{3} }$$

Collecting the black powers of $e$ gives the power $-t^2/2$ while collecting the blue powers (coming from the perturbations) gives

$$\sum_{i=1}^n \color{blue}{\sqrt{|p_it/\sqrt{n}|}(-1+i\operatorname{sgn}(p_it/\sqrt{n}))} = \sqrt{|t|}(-1+i\operatorname{sgn}(t))\frac{\sum_{i=1}^n \sqrt{p_i}}{n^{1/4}}\tag{4}$$

because $n$ and all the $p_i$ are positive. Since $|-1 + i\operatorname{sgn}(t)| \le \sqrt{2}$, for any fixed $t$ the value of $(4)$ goes to zero as $n$ increases provided $\sum_{i=1}^n\sqrt{p_i} = o(n^{-1/4}).$ One way to make this happen is to make the sum of the $\sqrt{p_i}$ converge: take $p_i = 2^{-2i}$, for instance. Then

$$\frac{1}{n^{1/4}} \sum_{i=1}^n \sqrt{p_i} \le \frac{1}{n^{1/4}} (1/2+1/4+\cdots+1/2^n+\cdots) = \frac{1}{n^{1/4}}\to 0.$$

Consequently, because the exponential is continuous at $0$, the blue terms $(3)$ converge to $e^0=1$: they do not affect the limit. We conclude $(\psi_{S_n})$ converges to $\psi_X$. Because this is the cf of the standard Normal distribution, Lévy's Continuity Theorem implies $S_n$ converges to a standard Normal distribution, QED.


Comments

The ideas displayed here can be generalized. We don't need the $X_n$ to be standard Normal; it suffices (by the usual Central Limit Theorem) that they are iid with zero mean and unit variance. It looks we have established an extension of the CLT: the distributions of means of a sequence of independent random variables, even those with infinite expectations and variances, can (when suitably standardized) converge to a standard Normal distribution, provided the "infinite part" of the random variables grows small sufficiently quickly.

$\endgroup$
2
  • $\begingroup$ How would you trick the mixture of random variables above to obtain $E[X_n]<\infty$ but $E[|X_n|]=\infty$? Thank you for any hint! $\endgroup$
    – Enrico
    Apr 8, 2023 at 9:20
  • $\begingroup$ @Enrico When $E[|X|]=\infty,$ $E[X]$ is undefined. $\endgroup$
    – whuber
    Apr 8, 2023 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.