5
$\begingroup$

Suppose I have a bivariate normal $(x,y)\sim \mathcal N(0,\Sigma)$ .

Is there an easy formula for the expectation $\mathrm E(x\mid y>0)$ ?

$\endgroup$
2
6
$\begingroup$

Let $\phi(\cdot)$ and $\Phi(\cdot)$ be the PDF and CDF of standard normal distribution, as usual.

Suppose $\Sigma=\left[\begin{matrix}\sigma_1^2&\rho \sigma_1\sigma_2\\\rho\sigma_1\sigma_2&\sigma_2^2\end{matrix}\right]$ is the dispersion matrix of $(X,Y)$.

By definition,

\begin{align} E\left[X\mid a<Y<b\right]&=\frac{E\left[XI(a<Y<b)\right]}{P(a<Y<b)} \\&=\frac1{P(a<Y<b)}\iint_{\mathbb R^2} x \mathbf1_{a<y<b}f_{X,Y}(x,y)\,\mathrm{d}x\,\mathrm{d}y \\&=\frac1{P(a<Y<b)}\int_a^b\int_{-\infty}^{\infty} xf_{X,Y}(x,y)\,\mathrm{d}x\,\mathrm{d}y \\&=\frac{1}{P(a<Y<b)}\int_a^b \left\{\int_{-\infty}^{\infty} xf_{X\mid Y}(x\mid y)\,\mathrm{d}x \right\}f_Y(y)\,\mathrm{d}y \\&=\frac{1}{P(a<Y<b)}\int_a^b E\left[X\mid Y=y\right]f_Y(y)\,\mathrm{d}y \end{align}

This of course is same as saying $$E\left[X\mid a<Y<b\right]=\int_a^b E\left[X\mid Y=y\right]f_{Y\mid a<Y<b}(y)\,\mathrm{d}y$$

Now $X$ conditioned on $Y=y$ has a $N\left(\frac{\rho\,\sigma_1}{\sigma_2}y,(1-\rho^2)\sigma_1^2\right)$ distribution, so that

$$E\left[X\mid Y=y\right]=\frac{\rho\,\sigma_1}{\sigma_2}y$$

In this case, we thus have

\begin{align} E\left[X\mid Y>0\right]&=\frac{\rho\,\sigma_1}{\sigma_2}\int_0^\infty \frac{yf_Y(y)}{P(Y>0)}\,\mathrm{d}y \\&=\frac{\rho\,\sigma_1}{\sigma_2}E\left[Y\mid Y>0\right] \end{align}

Finally,

\begin{align} E\left[Y\mid Y>0\right]&=\frac{1}{P(Y>0)}\int_0^\infty \frac{y}{\sigma_2}\phi\left(\frac{y}{\sigma_2}\right)\mathrm{d}y \\&=2\sigma_2\int_0^\infty t\phi(t)\,\mathrm{d}t \\&=2\sigma_2\int_0^\infty (-\phi'(t))\,\mathrm{d}t \\&=2\sigma_2\,\phi(0) \\&=\sqrt{\frac{2}{\pi}}\sigma_2 \end{align}

Hence for $(X,Y)\sim N(\mathbf 0,\Sigma)$, we have the simple formula

$$\boxed{E\left[X\mid Y>0\right]=\sqrt{\frac{2}{\pi}}\rho\,\sigma_1}$$

A general formula where the means of $X$ and $Y$ are not zero, and $Y$ ranging from some $a$ to $b$ can also be found in a similar manner. That formula would involve $\phi$ and $\Phi$, whereas for the current one, we get the values at $\phi$ and $\Phi$ directly.

$\endgroup$
3
  • $\begingroup$ You can write the answer in terms of the components of $\Sigma$ alone: it's simple. $\endgroup$ – whuber Jul 13 '18 at 23:29
  • $\begingroup$ @whuber For the given question, yes we get the answer in terms of the elements of $\Sigma$ only. What I was referring to before the edit was the general formula, which would naturally involve the pdf and cdf of a standard normal variable. $\endgroup$ – StubbornAtom Jul 14 '18 at 7:05
  • $\begingroup$ thanks for completing the answer, I posted mine below before seeing yours $\endgroup$ – andy Jul 14 '18 at 10:51
5
$\begingroup$

Let's generalize the question so that a key idea can be revealed.

Let $Y$ and $R$ be independent random variables. Define $$X=f(Y)+R$$ for a specified (measurable) function $f$. For any $a\lt b,$ what is the conditional expectation $$E[X\mid a\le Y \le b]?$$

(This states that $f$ is the regression of $X$ on $Y$ with i.i.d. additive errors.)

The independence assumption makes this an easy question to answer, because the random variables $f(Y)$ and $R$ will still be independent, whence (by "taking out what is known")

$$E[X\mid a\le Y \le b] = E[f(Y)+R\mid a\le Y \le b] = E[f(Y)\mid a \le Y \le b] + E[R].\tag{*}$$

This is very general. Let's specialize to the case $E[R]=0$ and $f$ is a linear transformation $$f(y) = \alpha\, y$$ for some constant number $\alpha.$ In this case $(*)$ simplifies to

$$E[X\mid a\le Y \le b] = \alpha\, E[Y\mid a \le Y \le b].\tag{**}$$

The right hand side has a direct expression when the distribution of $Y,$ $F_Y,$ has a density $f_y,$ for then (by the definitions of conditional probability and expectation)

$$E[Y\mid a \le Y \le b] = \frac{1}{F(b)-F(a)}\int_a^b y f_y(y) \mathrm{d}y.$$


The result is now immediate and obvious to anyone who has studied bivariate regression. (A good non-mathematical reference with all the necessary details is Freedman, Pisani, and Purves, Statistics [any edition].) The next section of this post is a review for those who might not have encountered this theory.

In the case of the question, where $(x,y)$ is bivariate Normal with mean $(0,0),$ standard deviations $\sigma_x$ and $\sigma_y,$ and correlation $\rho,$ we know that the distribution of $(x,y)$ is the same as the distribution of $(X,Y)$ constructed as above where $Y$ has a Normal$(0,\sigma_y^2)$ distribution and $R$ independently has a Normal$(0,\tau^2)$ distribution (with $\tau$ to be found). The proof of this lies in the observations that

  1. Because $Y$ and $R$ are independent, the variance of $X = \rho\,Y+R$ equals $$\rho^2 \operatorname{Var}(Y) +\operatorname{Var}(R) =\rho^2\sigma^2_y + \tau^2.$$ Consequently, if we set $$\tau^2 = \sigma_x^2 - \rho^2\sigma_y^2,$$ the variance of $X$ will equal the variance of $x.$

  2. The covariance $(X,Y)$ is $$\operatorname{Cov}(X,Y) = \operatorname{Cov}(\alpha\,Y+R,Y) = \alpha\operatorname{Var}(Y) = \alpha\,\sigma_y^2.$$ If we set $$\alpha = \rho\frac{\sigma_x}{\sigma_y}$$ then the correlation of $X$ and $Y$ will be $$\rho(X,Y) = \frac{\operatorname{Cov}(X,Y)}{\sigma_x\sigma_y} = \frac{\alpha\,\sigma_y^2}{\sigma_x\sigma_y} = \frac{\rho\frac{\sigma_x}{\sigma_y} \sigma_y^2}{\sigma_x\sigma_y} = \rho.$$

Observations (1) and (2) establish that $(X,Y)$ and $(x,y)$ have the same moments through second order, which means they have identical bivariate Normal distributions.

Applying the main result $(**)$ gives

$$E[X\mid a\le Y \le b] = \alpha\, E[Y\mid a \le Y \le b] = \rho\frac{\sigma_x}{\sigma_y}E[Y\mid a \le Y \le b].$$

We can go a little further towards simplification by recognizing that $Y/\sigma_y$ is a standard Normal variate. Thus,

$$E[X\mid a\le Y \le b] = \rho\,\sigma_x E\left[\frac{Y}{\sigma_y}\mid \frac{a}{\sigma_y} \le \frac{Y}{\sigma_y} \le \frac{b}{\sigma_y} \right].$$

This shows that for fixed $a,b,$ the answer is directly proportional to $\rho\, \sigma_x.$ Evaluating the constant is a matter of manipulating integrals of a standard Normal variable, but doing so is primarily an exercise in Calculus rather than of statistical interest.


I will conclude with the expression that I actually began with when solving this problem. Notice that the answer can also be written $$E\left[\frac{X}{\sigma_x}\mid a\le Y \le b\right] = \rho\, E\left[\frac{Y}{\sigma_y}\mid \frac{a}{\sigma_y} \le \frac{Y}{\sigma_y} \le \frac{b}{\sigma_y} \right].$$

The left hand side is the conditional expectation of a standard Normal variate $X/\sigma_x$ while the right hand side is just $\rho$ times the expectation of a truncated standard Normal variate $Y/\sigma_y.$

If you get used to standardizing variables automatically--which amounts to choosing a particularly convenient unit of measurement for them--then simply by visualizing the regression of $X$ against $Y$ you will say to yourself

oh yes, since after standardization $X$ is just a multiple $\rho$ of $Y$, then the conditional expectation of $X$ must be $\rho$ times the corresponding expectation of $Y.$

That is the heart of the matter.

$\endgroup$
1
  • 2
    $\begingroup$ This provides good intuition rather than merely calculating the conditional expectation directly as I would have done. $\endgroup$ – StubbornAtom Jul 14 '18 at 21:33
2
$\begingroup$

I am posting the complete answer for reference: \begin{eqnarray*} E(X|a<Y<b) &= &\int_a^bE(X|Y=y)f_{Y|a<Y<b}(y)dy \\ & = &\rho_{xy}\frac{\sigma_x}{\sigma_y}\int_a^bf_{Y|a<Y<b}(y)dy \\ & = &\rho_{xy}\frac{\sigma_x}{\sigma_y}\sigma_y\frac{\phi(a)-\phi(b)}{\Phi(b)-\Phi(a)} \\ & = &\frac{\sigma_{xy}}{\sigma_y}\frac{\phi(a)-\phi(b)}{\Phi(b)-\Phi(a)} ,\end{eqnarray*} where $\rho_{xy} = E(xy) / (\sigma_x\sigma_y) =\sigma_{xy} / (\sigma_x \sigma_y)$.

The second-to-last step is derived from the properties of the univariate truncated normal distribution, easily found on its wikipedia page. A reference for the first step is here

With $a=0,b=\infty$ we have $$ E(X,Y>0) = \rho_{xy}\sigma_x\frac{\phi(0)}{\Phi(0)}= \rho_{xy}\sigma_x\cdot \frac{2}{\sqrt{2\pi}}= \frac{\sigma_{xy}}{\sigma_y}\cdot \frac{2}{\sqrt{2\pi}} $$ Thanks for pointing in the right direction.

$\endgroup$
3
  • 1
    $\begingroup$ There must be a typographical error because the $\sigma_y/\sigma_y$ term cancels, leaving the answer independent of $\sigma_y$ altogether. $\endgroup$ – whuber Jul 14 '18 at 14:04
  • 2
    $\begingroup$ But $\rho_{xy}$ contains $\sigma_y$ $\endgroup$ – andy Jul 14 '18 at 15:36
  • 1
    $\begingroup$ Don't you have to standardize $a$ and $b$? $\endgroup$ – Waldir Leoncio Jan 3 '19 at 11:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.