# CLT with random variables that are not integrable

Exercise 15.5.1 from Klenke's "Probability Theory: A Comprehensive Course" reads as follows. Find a sequence $X_1, X_2, \ldots$ of independent real random variables with $\mathbb{E}[|X_n|]=\infty$ for all $n\in \mathbb{N}$ such that $$\frac{X_1+ \cdots + X_n}{\sqrt{n}} \stackrel{n \to \infty}{\Longrightarrow} N(0,1).$$ I'm not sure how this is possible if the mean isn't even defined on this case. All the cases I can think of of variables with undefined mean do not satisfy a Central Limit Theorem with a $\sqrt{n}$ scaling. Any help is appreciated.

• Hint: Reduce the concepts to their basic elements. Since the Normal is at the heart of the CLT, start with iid standard Normal $X_i$. Then modify them so that (a) each has infinite expected absolute value while (b) the amount of modification decreases rapidly with $n$, so that in the limit division by $\sqrt{n}$ "kills off" the modified part. A simple way to modify a random variable is to add another one to it.
– whuber
Dec 8, 2016 at 14:41
• @whuber Is your point that the random variables are not said to be identically distributed? If they were i.i.d. I would agree with the OP. Dec 8, 2016 at 15:57
• @whuber Also does the answer require that only Xn has the infinite mean. Dec 8, 2016 at 16:01
• I guess the "for all n" means that all the Xi have infinite mean absolute value. I am trying to be careful here. I want to understand your hint but not say something that gives away the solution to the OP. Dec 8, 2016 at 16:10
• @Michael If they were iid then the result would not be true. The idea is to make the part of $X_n$ that causes the infinite expectation to have less and less chance of occurring as $n$ increases. A good way to do that is with a mixture.
– whuber
Dec 8, 2016 at 16:49

Among the many ways to solve this one, constructing the sequence by perturbing a standard Normal variable seems like the simplest and most elegant.

At the end I comment on the connection with the Central Limit Theorem.

## Characteristic Functions

Allow me a digression before I present a solution. The inspiration for the technique that will be used comes from the idea that there is more than one way to describe the distribution of any random variable $X$. Commonest, and most direct, is its distribution function $F_X(x)=\Pr(X\le x)$. An indirect but extremely useful alternative is its characteristic function

$$\psi_X(t) = E\left[e^{itX}\right] = E\left[\cos(t X)\right] + i\, E\left[\sin(t X)\right].$$

Because $|e^{itX}|=1$ for all $t$, $\psi_F$ is defined for any distribution $F$ (and its values for all $t$ cannot exceed $1$ in size). Moreover, $X$ and $Y$ have the same distribution if and only if they have the same characteristic function. Even better is Lévy's Continuity Theorem: A sequence $X_n$ converges in distribution to a random variable $X$ if and only if for every $t$ the sequence $\phi_{X_n}(t)$ converges to a value $\psi(t)$ and the function $\psi$ is continuous at $0$. (All characteristic functions are continuous at $0$.) In that case, $\psi$ is the characteristic function of $X$.

Another of the lovely properties enjoyed by characteristic functions is their relationship with linear combinations: when $X$ and $Y$ are random variables (on the same probability space and $\alpha$ and $\beta$ are real numbers,

$$\psi_{\alpha X+\beta Y}(t) = \psi_X(\alpha t)\psi_Y(\beta t).\tag{1}$$

This makes characteristic functions (cfs) a suitable tool for studying perturbations of random variables $X$ achieved by adding tiny amounts of other random variables $Y$ to them: that is, random variables of the form $X+\beta Y$ for $|\beta|$ small.

## Solution

### Construction of a sequence

Let's construct a solution by starting with a standard Normal variable $Z$ and forming an independent sequence $Z_1, Z_2, \ldots, Z_n, \ldots$ with the same distribution as $Z$. This obviously has the limiting property we want: the means are all standard Normal, so in the limit the mean is standard Normal.

Its cf is

$$\psi_Z(t) = e^{-t^2/2}.\tag{2}$$

For the perturbations, pick some random variable $Y$ with infinite expectation. It will be convenient for $Y$ to have a cf that's easy to work with. I would like to suggest the Lévy Distribution (aka Stable Distribution with $\alpha=1/2,\ \beta=1$ or Inverse Gamma$(1/2,1/2)$ distribution) for which

$$\psi_Y(t) = e^{-\sqrt{|t|}\,(1 - i \operatorname{sgn}(t))}.$$

(For $t\gt 0$, $\operatorname{sgn}(t)=1$; for $t \lt 0,$ $\operatorname{sgn}(t)=-1$.)

This distribution is supported on $(0,\infty)$ and has no finite moments.

To this sequence of standard normal variables $(Z_n)$ let's add ever-smaller positive multiples of $Y$. (Positivity is unnecessary but it makes working with the $\operatorname{sgn}$ function easier.) Let the sequence of multiples be $p_1,p_2,p_3,\ldots,$ to be determined. Thus, the sequence of random variables is defined to be $$X_n=Z_n + p_n Y_n$$ where $(Y_n)$ is an iid sequence of random variables with the same distribution as $Y$.

### Intuition

What we need to worry about is whether the perturbations are so bad that they ruin the convergence to a standard Normal distribution. To those with experience with such heavy-tailed distributions, this is a real concern: there will always be some positive probability that the little bit of $Y_n$ added into $Z_n$ will occasionally introduce such a whopping big outlier that it overwhelms the partial sum $S_n$. The entire reason for using characteristic functions is to demonstrate this will not happen in the long run, provided we reduce the amount of perturbation (the $p_n$) sufficiently rapidly.

### Formal calculations

First, $X_n$ has infinite expectation because

$$E[X_n] = E[Z_n + p_n Y_n] = E[Z] + p_n E[Y] = p_n E[Y]$$

must be infinite since $E[Y]$ is infinite. Thus this sequence $(X_n)$ satisfies all the requirements of the problem.

Let's turn to the analysis of the partial means. Repeated application of $(1)$ to the partial mean

$$S_n = \frac{X_1 + X_2 + \cdots + X_n}{\sqrt{n}}$$

gives

\eqalign{ \psi_{S_n}(t) &= \left[e^{-(t/\sqrt{n})^2/2}\color{Blue}{\psi_Y(p_1 t/\sqrt{n})}\right] \cdots \left[e^{-(t/\sqrt{n})^2/2}\color{Blue}{\psi_Y(p_n t/\sqrt{n})}\right] \\ &= \left[e^{-(t/\sqrt{n})^2/2} \cdots e^{-(t/\sqrt{n})^2/2}\right] \left[\color{Blue}{\psi_Y(p_1t/\sqrt{n}) \cdots \psi_Y(p_nt/\sqrt{n})}\right] \\ &= e^{-t^2/(2n) - t^2/(2n) - \cdots - t^2/(2n)}\quad \color{Blue}{e^{\sqrt{|p_1t/\sqrt{n}|}(-1+i\operatorname{sgn}(p_1t/\sqrt{n})} \cdots e^{\sqrt{|p_nt/\sqrt{n}|}(-1+i\operatorname{sgn}(p_nt/\sqrt{n})} }.\tag{3} }

Collecting the black powers of $e$ gives the power $-t^2/2$ while collecting the blue powers (coming from the perturbations) gives

$$\sum_{i=1}^n \color{blue}{\sqrt{|p_it/\sqrt{n}|}(-1+i\operatorname{sgn}(p_it/\sqrt{n}))} = \sqrt{|t|}(-1+i\operatorname{sgn}(t))\frac{\sum_{i=1}^n \sqrt{p_i}}{n^{1/4}}\tag{4}$$

because $n$ and all the $p_i$ are positive. Since $|-1 + i\operatorname{sgn}(t)| \le \sqrt{2}$, for any fixed $t$ the value of $(4)$ goes to zero as $n$ increases provided $\sum_{i=1}^n\sqrt{p_i} = o(n^{-1/4}).$ One way to make this happen is to make the sum of the $\sqrt{p_i}$ converge: take $p_i = 2^{-2i}$, for instance. Then

$$\frac{1}{n^{1/4}} \sum_{i=1}^n \sqrt{p_i} \le \frac{1}{n^{1/4}} (1/2+1/4+\cdots+1/2^n+\cdots) = \frac{1}{n^{1/4}}\to 0.$$

Consequently, because the exponential is continuous at $0$, the blue terms $(3)$ converge to $e^0=1$: they do not affect the limit. We conclude $(\psi_{S_n})$ converges to $\psi_X$. Because this is the cf of the standard Normal distribution, Lévy's Continuity Theorem implies $S_n$ converges to a standard Normal distribution, QED.

The ideas displayed here can be generalized. We don't need the $X_n$ to be standard Normal; it suffices (by the usual Central Limit Theorem) that they are iid with zero mean and unit variance. It looks we have established an extension of the CLT: the distributions of means of a sequence of independent random variables, even those with infinite expectations and variances, can (when suitably standardized) converge to a standard Normal distribution, provided the "infinite part" of the random variables grows small sufficiently quickly.
• How would you trick the mixture of random variables above to obtain $E[X_n]<\infty$ but $E[|X_n|]=\infty$? Thank you for any hint! Apr 8, 2023 at 9:20
• @Enrico When $E[|X|]=\infty,$ $E[X]$ is undefined.