# Upper Bound of MGF for a non-negative random variable with bounded variance

Let $$X$$ be a non-negative random variable with finite variance. It is obvious that its MGF $$E[e^{-\lambda(X-E[X])}]$$ exists for $$\lambda > 0$$.

How to prove that $$E[e^{-\lambda(X-E[X])}] \le \exp(\lambda^2 E[X^2]/2)$$ for $$\lambda > 0$$?

• $E[e^{-\lambda(X - E[X])}]$ is not MGF: $E[e^{\lambda X}]$ is. May 17 at 20:53

The inequality $$e^{-u} \leq 1 - u + u^2 /2$$ holds for any $$u \geq 0$$. Replacing $$u$$ by $$\lambda X$$ and taking the expectation we get $$\mathbb{E}[e^{-\lambda X}] \leqslant 1 - \lambda \,\mathbb{E}(X) + \lambda^2\,\mathbb{E}(X^2) /2.$$ Now since $$1 + v \leq e^v$$ for any $$v$$, by choosing $$v := - \lambda \,\mathbb{E}(X) + \lambda^2 \mathbb{E}(X^2)/2$$ $$\mathbb{E}[e^{-\lambda X}] \leq \exp\left \{-\lambda \,\mathbb{E}(X) + \lambda^2 \,\mathbb{E}(X^2)/2 \right\}$$ which is the result in OP.

• Great answer, which fixes my previous answer's hole nicely! (+1) May 18 at 13:20
• Short and sweet, very nice :) (+1) May 18 at 14:15

Here is a proof adapted from John Duchi's lecture notes, but it's off by a factor of $$2$$ if you don't mind : We will assume throughout that $$X$$ has an MGF defined on a neighborhood of $$0$$, such that all of its moments exist (as finite variance is not sufficient).

Let $$X'$$ be an independent copy of $$X$$, i.e. $$X'$$ has the same distribution as $$X$$ while being independent of $$X$$. We have $$E[e^{-\lambda(X-E[X])}] = E_X[e^{-\lambda(X-E_{X'}[X'])}] \le E_XE_{X'}[e^{-\lambda(X-X')}]$$ Where we have applied Jensen's inequality to $$e^{\lambda E_{X'}[X']}:= f(E_{X'}[X'])$$.

Now for convenience let $$Y:= X-X'$$. The trick is to notice that $$Y$$ and $$-Y$$ are identically distributed, which implies that $$Y$$ and $$S\cdot Y$$ are identically distributed when $$S$$ is an independent Rademacher random variable, i.e. when $$S$$ only takes value $$+1$$ and $$-1$$ with equal probability.

This implies that \begin{align*} E_Y[e^{-\lambda Y}] &= E_YE_S[e^{-\lambda S\cdot Y}] \\ &= E_Y[E_S[e^{-\lambda S\cdot Y}\mid Y]]\\ &= E_Y[\cosh(-\lambda Y)] =E_Y[\cosh(\lambda Y)]\end{align*}

Now assume that $$Y$$ is supported on some interval $$[-a,a]$$. By Taylor's theorem, we know that for all $$Y(\omega)$$, there exists $$\xi\in (0,Y)\cup (Y,0)$$ such that $$\cosh(\lambda Y) = 1 + \frac{\lambda^2Y^2}{2} + \frac{\lambda^3Y^3}{6}\sinh(\lambda\xi(Y)) \le 1 + \frac{\lambda^2Y^2}{2} + \frac{\lambda^3Y^3}{6}\sinh(\lambda a)$$

Because $$Y$$ is symmetric, we have that $$E[Y^3]=0$$, hence taking expectation we find $$E_Y[\cosh(\lambda Y)] \le 1 + \frac{\lambda^2}{2}E_Y[Y^2]\le \exp\left(\frac{\lambda^2}{2}E_Y[Y^2]\right)$$ Where we have used the well known $$1 + x \leq e^{x}$$. All that is left is to observe that $$E_Y[Y^2] = E_Y[X^2 + (X')^2 - 2XX'] = 2E[X^2] - 2E[X]^2$$ and it follows that $$E[e^{-\lambda(X-E[X])}] \le \exp\left(\lambda^2 (E[X])^2\right)$$

Which is the desired result up to a factor $$2$$.

Now in the general case where $$Y$$ is not compactly supported, you can construct a sequence $$Y_n = (-n)\vee(Y\wedge n) \in [-n,n]$$ which is bounded and converges pointwise to $$Y$$. It then follows from Fatou's lemma that \begin{align*} E_Y[\cosh(\lambda Y)]&\le \lim\inf_n E_{Y_n}[\cosh(\lambda Y_n)] \\ &\le \lim\inf_n \exp\left(\frac{\lambda^2}{2}E_{Y_n}[Y_n^2]\right)\\ &\le\exp\left(\frac{\lambda^2}{2}E_Y[Y^2]\right)\end{align*}. From which the same conclusion follows.

• The inequality $Y^3 \sinh(\lambda \xi) \leq Y^3 \sinh(\lambda a)$ does not seem so easy since $Y$ can be negative and $\xi$ depends on $Y$. Maybe a justification could help?
• @Yves : that's because I picked $\xi$ between $0$ and $Y$, so $Y^3 \sinh(\lambda \xi)$ is always non-negative, and then the upper bound follows from the fact that $Y\le a$ almost surely and that $\sinh$ is non-decreasing. May 18 at 10:47
• When $Y$ is negative I get $-\sinh(\lambda a) \leq \sinh(\lambda \xi) \leq 0$ because $-a \leq \xi \leq 0$ and $\sinh(-\lambda a) = - \sin(\lambda a)$. But then by multiplying by $Y^3 \leq 0$ I get $0 \leq Y^3 \sinh(\lambda \xi) \leq -Y^3 \sinh(\lambda a)$ which still puzzles me.
• $-Y^3\sinh(\lambda a) = Y^3\sinh(-\lambda a) \le Y^3\sinh(\lambda a)$ so seems like there is no problem... Or am I missing something ? May 18 at 14:13