4
$\begingroup$

I'm trying to follow a proof in the Adversarial Spheres preprint on arXiv. The proof requires the computation of the integral in Appendix F, page 14:

$$\mathbf{E}\left[\max\left(\sqrt{2}\left(\frac{\alpha}{\sqrt{n}}-Z\right),0\right)\right],\quad Z\sim\mathcal{N}\left(0,\frac{1}{n}\right)$$

where $\alpha>0$. According to the paper, this integral should be $O\left(\frac{\alpha}{\sqrt{n}}\right)$.Now, since

$$\sqrt{n}Z=X\sim\mathcal{N}\left(0,1\right)$$

I can rewrite the expectation as

$$\mathbf{E}\left[\max\left(\sqrt{2}\left(\frac{\alpha-X}{\sqrt{n}}\right),0\right)\right],\quad X\sim\mathcal{N}(0,1)$$

Using the properties of $\max(x,0)$ and expectation, I rewrite

$$\mathbf{E}_{X\sim\mathcal{N}(0,1)}\left[\max\left(\sqrt{2}\left(\frac{\alpha-X}{\sqrt{n}}\right),0\right)\right] = \sqrt{\frac{2}{n}} \mathbf{E}_{X\sim\mathcal{N}(0,1)}\left[\max\left(\alpha-X,0\right)\right]$$

I'm left with computing

$$\mathbf{E}_{X\sim\mathcal{N}(0,1)}\left[\max\left(\alpha-Y,0\right)\right]=\int_{-\infty}^{\infty}\max(\alpha-x,0)\frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}dx=\int_{-\infty}^{\alpha}(\alpha-x)\frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}dx$$

This integral can be split into

$$\alpha\int_{-\infty}^{\alpha}\frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}dx=\alpha\Phi(\alpha)$$

and

$$-\int_{-\infty}^{\alpha}\frac{x}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}dx=\int_{-\alpha}^{\infty}\frac{x}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}dx$$

Now, this last integral should be equal to the conditional expectation

$$\mathbf{E}[X|-\alpha<X]$$

(right?). Thus I should have

$$\int_{-\alpha}^{\infty}\frac{x}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}dx=\frac{\phi(-\alpha)}{1-\Phi(-\alpha)}$$

Thus, finally I should have

$$\mathbf{E}_{Z\sim\mathcal{N}\left(0,\frac{1}{n}\right)}\left[\max\left(\sqrt{2}\left(\frac{\alpha}{\sqrt{n}}-Z\right),0\right)\right] = \sqrt{\frac{2}{n}}\left(\alpha\Phi(\alpha)+\frac{\phi(-\alpha)}{1-\Phi(-\alpha)}\right)$$

Questions:

  1. Is this correct?
  2. Can I conclude that, since $\Phi(\alpha)$ and $\frac{\phi(-\alpha)}{1-\Phi(-\alpha)}$ are bounded for $\alpha>0$, the integral is $O\left(\frac{\alpha}{\sqrt{n}}\right)$ as stated in the paper? Specifically, since $\Phi(\alpha) \to 1$ and $\frac{\phi(-\alpha)}{1-\Phi(-\alpha)} \to 0$ as $\alpha \to \infty$, I have that the integral is well-approximated by $\sqrt{\frac{2}{n}}\alpha$, as shown below:

    a <- seq(0, 10, len = 1000)
    y <- a*pnorm(a)+dnorm(-a)/(1-pnorm(-a))
    plot(a, y, type = "l")
    abline(a = 0, b = 1, col = "red")
    

    enter image description here

$\endgroup$
2
  • 2
  • $\begingroup$ hi @whuber, thanks for the comment, it's been a while since the last time we met on the site! The first & second questions don't seem exactly relevant, but the third one is right on the money - just substituting $X=-Y$ in your proof for $\mathbf{E}(X|X≤\alpha)$ leads immediately to my formula. Let me know if you would like to write an answer and get the well-deserved reputation, otherwise I'll self-answer my own question and accept my answer. $\endgroup$
    – DeltaIV
    Sep 26, 2018 at 9:21

1 Answer 1

3
$\begingroup$

Adapting the answer by whuber to this question:

Expected value of x in a normal distribution, GIVEN that it is below a certain value

we have that for a standard normal $X$

$$\mathbf{E}[X|X \le \alpha] = - \frac{\phi\left(\alpha\right)}{\Phi\left(\alpha\right)}$$

Now, setting $Y=-X$, and noting that $Y\sim\mathcal{N}(0,1) $, we have

$$\mathbf{E}[X|X \le \alpha]=\mathbf{E}[-Y|-Y \le \alpha] = - \mathbf{E}[Y|-\alpha \le Y]=-\frac{\phi\left(\alpha\right)}{\Phi\left(\alpha\right)}\implies\mathbf{E}[Y|-\alpha \le Y]=\frac{\phi\left(\alpha\right)}{\Phi\left(\alpha\right)}$$

Since

$$\frac{\phi\left(\alpha\right)}{\Phi\left(\alpha\right)}=\frac{\phi\left(-\alpha\right)}{1-\Phi\left(-\alpha\right)}$$

my formula

$$\mathbf{E}_{Z\sim\mathcal{N}\left(0,\frac{1}{n}\right)}\left[\max\left(\sqrt{2}\left(\frac{\alpha}{\sqrt{n}}-Z\right),0\right)\right] = \sqrt{\frac{2}{n}}\left(\alpha\Phi(\alpha)+\frac{\phi(-\alpha)}{1-\Phi(-\alpha)}\right)$$

is proved and it can be simplified to

$$\mathbf{E}_{Z\sim\mathcal{N}\left(0,\frac{1}{n}\right)}\left[\max\left(\sqrt{2}\left(\frac{\alpha}{\sqrt{n}}-Z\right),0\right)\right] = \sqrt{\frac{2}{n}}\left(\alpha\Phi(\alpha)+\frac{\phi(\alpha)}{\Phi(\alpha)}\right)$$

thus the answer to my questions is 1) yes and 2) yes.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.