The following comes From Martin Wainwright's book on High-Dimensional Statistics, page 41 on Lipschitz functions of Gaussian variables.

It first begins by the following Lemma:

Suppose $f:\mathbb{R}^n\to\mathbb{R}$ is differentiable. Then for any convex function $\phi:\mathbb{R}\to\mathbb{R}$, we have \begin{equation} \mathbb{E}[\phi(f(X)-\mathbb{E}[f(X)])]\leq\mathbb{E}\left[\phi\left(\frac{\pi}{2}\langle\nabla f(X),Y\rangle\right)\right] \end{equation} where $Y,X\sim N(0,I_{n})$ are standard multivariate Gaussian, and independent.

It then goes on to show this:

For any $\lambda\in\mathbb{R}$, applying the inequality above to the convex function $t\to\exp(\lambda t)$ yields \begin{align} \mathbb{E}_X\left[\exp\left(\lambda\{f(X)-\mathbb{E}[f(X)]\}\right)\right]&\leq \mathbb{E}_{X,Y}\left[\exp\left(\frac{\lambda\pi}{2}\langle \nabla f(X),Y \rangle\right)\right]\\ &=\mathbb{E}_X\left[\exp\left(\frac{\lambda^2\pi^2}{8}\lVert\nabla f(X)\rVert_2^2\right)\right] \end{align} where we have used the independence of $X$ and $Y$ to first take the expectation over $Y$, and the fact that $\langle Y,\nabla f(X)\rangle$ is a zero-mean Gaussian variable with variance $\lVert \nabla f(x)\rVert_2^2$.

I do not follow the equality in the second line of the most recent lines of the equations. Some clarification here would be very much appreciated.

  • $\begingroup$ I think you must have omitted a "$\lambda$" from the inequality at the end. You might find this problem to be more accessible by first considering the case $n=1.$ $\endgroup$
    – whuber
    Mar 16, 2021 at 17:50
  • 1
    $\begingroup$ Good point. I have added the lambda. I shall give it a try by considering n=1. Thanks. $\endgroup$
    – Carl
    Mar 16, 2021 at 17:54


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.