14
$\begingroup$

This question arises from the one asked here about a bound on moment generating functions (MGFs).

Suppose $X$ is a bounded zero-mean random variable taking on values in $[-\sigma, \sigma]$ and let $G(t) = E[e^{tX}]$ be its MGF. From a bound used in a proof of Hoeffding's Inequality, we have that $$G(t) = E[e^{tX}] \leq e^{\sigma^2t^2/2}$$ where the right side is recognizable as the MGF of a zero-mean normal random variable with standard deviation $\sigma$. Now, the standard deviation of $X$ can be no larger than $\sigma$, with the maximum value occurring when $X$ is a discrete random variable such that $P\{X = \sigma\} = P\{X = -\sigma\} = \frac{1}{2}$. So, the bound referred to can be thought of as saying that the MGF of a zero-mean bounded random variable $X$ is bounded above by the MGF of a zero-mean normal random variable whose standard deviation equals the maximum possible standard deviation that $X$ can have.

My question is: is this a well-known result of independent interest that is used in places other than in the proof of Hoeffding's Inequality, and if so, is it also known to extend to random variables with nonzero means?

The result that prompts this question allows asymmetric range $[a,b]$ for $X$ with $a < 0 < b$ but does insist on $E[X] = 0$. The bound is $$G(t) \leq e^{t^2(b-a)^2/8} = e^{t^2\sigma_{max}^2/2}$$ where $\sigma_{\max} = (b-a)/2$ is the maximum standard deviation possible for a random variable with values restricted to $[a,b]$, but this maximum is not attained by zero-mean random variables unless $b = -a$.

$\endgroup$
  • 5
    $\begingroup$ Random variables that satisfy bounds on the mgf like the one you quote are called subgaussian random variables. They play a central role, e.g., in nonasymptotic random matrix theory and some associated results in compressed sensing. See, e.g., the link in the answer here. (This obviously doesn't speak to your particular question; but, it is of a related nature.) $\endgroup$ – cardinal Jan 16 '12 at 3:59
5
$\begingroup$

I can't answer the first part of your question, but as for extending it to random variables with nonzero means...

First, note that any r.v. $Z$ with finite range $[a+\mu,b+\mu]$ and (necessarily finite) mean $\mu$ can be transformed into an r.v. $X = Z-\mu$ that is, of course, zero mean with range $[a,b]$ (thus satisfying the conditions in your problem statement). The transformed variate has m.g.f. $\phi_X(t) = \exp\{-\mu t\}\phi_Z(t)$ (by basic properties of the m.g.f.) Multiplying both sides by $\exp\{\mu t\}$ and applying the inequality gives:

$\phi_Z(t) = \exp\{\mu t\}\phi_X(t) \leq \exp\{\mu t\}\exp\{t^2\sigma^2_\text{max}/2\} = \exp\{\mu t + t^2\sigma^2_\text{max}/2\} $

Not surprisingly, the m.g.f. of a Normal random variable with the same mean and standard deviation equal to $\sigma_\text{max}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.