Bound on moment generating function

Question

This question arises from the one asked here about a bound on moment generating functions (MGFs).

Suppose $X$ is a bounded zero-mean random variable taking on values in $[-\sigma, \sigma]$ and let $G(t) = E[e^{tX}]$ be its MGF. From a bound used in a proof of Hoeffding's Inequality, we have that $$G(t) = E[e^{tX}] \leq e^{\sigma^2t^2/2}$$ where the right side is recognizable as the MGF of a zero-mean normal random variable with standard deviation $\sigma$. Now, the standard deviation of $X$ can be no larger than $\sigma$, with the maximum value occurring when $X$ is a discrete random variable such that $P\{X = \sigma\} = P\{X = -\sigma\} = \frac{1}{2}$. So, the bound referred to can be thought of as saying that the MGF of a zero-mean bounded random variable $X$ is bounded above by the MGF of a zero-mean normal random variable whose standard deviation equals the maximum possible standard deviation that $X$ can have.

My question is: is this a well-known result of independent interest that is used in places other than in the proof of Hoeffding's Inequality, and if so, is it also known to extend to random variables with nonzero means?

The result that prompts this question allows asymmetric range $[a,b]$ for $X$ with $a < 0 < b$ but does insist on $E[X] = 0$. The bound is $$G(t) \leq e^{t^2(b-a)^2/8} = e^{t^2\sigma_{max}^2/2}$$ where $\sigma_{\max} = (b-a)/2$ is the maximum standard deviation possible for a random variable with values restricted to $[a,b]$, but this maximum is not attained by zero-mean random variables unless $b = -a$.

Answer 1

I can't answer the first part of your question, but as for extending it to random variables with nonzero means...

First, note that any r.v. $Z$ with finite range $[a+\mu,b+\mu]$ and (necessarily finite) mean $\mu$ can be transformed into an r.v. $X = Z-\mu$ that is, of course, zero mean with range $[a,b]$ (thus satisfying the conditions in your problem statement). The transformed variate has m.g.f. $\phi_X(t) = \exp\{-\mu t\}\phi_Z(t)$ (by basic properties of the m.g.f.) Multiplying both sides by $\exp\{\mu t\}$ and applying the inequality gives:

$\phi_Z(t) = \exp\{\mu t\}\phi_X(t) \leq \exp\{\mu t\}\exp\{t^2\sigma^2_\text{max}/2\} = \exp\{\mu t + t^2\sigma^2_\text{max}/2\} $

Not surprisingly, the m.g.f. of a Normal random variable with the same mean and standard deviation equal to $\sigma_\text{max}$.

Answer 2

Since $e^{ t x}$ is a convex function, by Jensen's inequality, we have $$e^{t X} \le \frac{\sigma+X}{2\sigma}e^{-t\sigma}+ \frac{\sigma-X}{2\sigma}e^{t\sigma}$$

Taking expectation of both sides of above inequality we get $$E[ e^{t X}] \le \frac{1}{2}e^{-t\sigma} + \frac{1}{2}e^{t\sigma}$$ where we used the zero-mean assumption on $X$. It remains to prove that $\frac{1}{2}(e^y+e^{-y})\le e^{y^2/2}$. (Then, replacing $y=t\sigma$ we arrive at $E[ e^{t X}] \le e^{t^2\sigma^2/2}$.)

Here is my proof for $$\quad \quad e^y+e^{-y}\le 2 e^{y^2/2} \quad \quad \text{for }y \in \mathbb{R} \quad (1)$$.

By Taylor expansion of $e^y$ we have $$e^y + e^{-y} = \sum_{n=0}^{\infty} \frac{y^n}{n!} + \sum_{n=0}^{\infty} \frac{(-1)^ny^n}{n!} =2\sum_{k=0}^{\infty} \frac{y^{2k}}{(2k)!} $$ $$2e^{y^2/2} = 2\sum_{k=0}^{\infty} \frac{y^{2k}}{2^k k!}$$ from which we get (1) since $(2k)! \ge 2^k k!$ (this can be shown by induction or Stirling bounds on k!).

س