6
$\begingroup$

If $\tilde{u}$ is a normally distributed random variable with mean $q$ and variance $\sigma$, and $s=\tilde{u}+\tilde{e}$ where $e$ is also normally distributed with mean 0 and variance $\sigma_s$. $\tilde{u}$ and $\tilde{e}$ are independent of each other. Why is it that $E[\tilde{u}|s\leq a]=q-\sigma\frac{f(a)}{F(a)}$, where $f(a)$ is the PDF of and $F(a)$ is the CDF of $s$ at $a$. This related question is the closest explanation I could find but the difference is that in the post $a$ is standardized.

$\endgroup$
10
  • $\begingroup$ Welcome to cv, r-learning-machine! Could you double check a few things: 1) what is $\tilde u$, 2) is $e$ independent of $u$ 3) is $F$ the cdf of s, or $G$? 4) what is the variance if $e$. $\endgroup$
    – Ute
    Commented Aug 20, 2023 at 19:54
  • $\begingroup$ Thank you for the comment. I have updated the post now so that these questions should be clear. $\endgroup$ Commented Aug 20, 2023 at 21:49
  • $\begingroup$ I have calculated a bit and think it should be $E[\tilde{u}|s\leq a]=q-\sigma^2\frac{f(a)}{F(a)}$ (replace $\sigma$ by $\sigma^2$). I might post an answer with more usual (general) symbols $X$ and $Y$ for the random variables, just because I make less typos than when using $\tilde u$ and $s$. $\endgroup$
    – Ute
    Commented Aug 21, 2023 at 14:46
  • 1
    $\begingroup$ Because $(\tilde u, s)$ has a Bivariate Normal distribution, you can find the answer worked out in detail at stats.stackexchange.com/questions/166273. It is not just in terms of a "standardized" $a:$ it is fully general. $\endgroup$
    – whuber
    Commented Aug 21, 2023 at 17:44
  • 1
    $\begingroup$ @whuber, would be interesting to see that if you have time. Sorry for the confusing 100x, a pun referring to "few hundred posts". $\endgroup$
    – Ute
    Commented Aug 21, 2023 at 22:32

2 Answers 2

4
$\begingroup$

Given the univariate formula provided at Expected value of x in a normal distribution, GIVEN that it is below a certain value, you can solve this mentally by choosing appropriate units of measurement and understanding the basics of regression.

Write $\sigma^2=\operatorname{Var}(\tilde u).$ Temporarily employ a unit of measurement for $s$ in which $E[s]=0$ and $\operatorname{Var}(s)=1.$ That is, $s$ has a standard Normal distribution with density $\phi$ and distribution $\Phi.$

We know from the basic theory of least squares regression that the conditional expectation of $\tilde u$ is a linear function of $s,$ $$E[\tilde u \mid s] = \beta s,$$

where $\beta\operatorname{Var}(s) = \operatorname{Var}(\tilde u)$ (in the temporary units).

Consequently

$$E[\tilde u \mid s \le a] = E[E[\tilde u\mid s] \mid s \le a] = \beta E[s \mid s \le a].$$

The problem has been reduced from a bivariate one to a univariate one involving only $s.$ The formula in the referenced thread states

$$E[s \mid s \le a] = - \frac{\phi(a)}{\Phi(a)}.$$

Converting back to the original units changes the right hand side to $-f(a)/F(a)$ where $f$ and $F$ are the density and distribution of $s,$ but the units calculus requires it to be further multiplied by $\operatorname{Var}(s)$ (because the left hand side is in units of $s$ and the right hand side is reciprocal units: $f$ is a density and $F$ is unitless.) Thus the net multiplier is $\beta\operatorname{Var}(s) = \beta\operatorname{Var}(\tilde u) = \sigma^2$ and, finally, we can return to the original units of measurement by adding the origin $q$ back in to produce

$$E[\tilde u \mid s \le a] = q - \sigma^2 \frac{f(a)}{F(a)}.$$

$\endgroup$
6
  • $\begingroup$ Thank you for your explanations @Ute and whuber. I am just trying to understand it fully. In my view, the conditional expectation should be $E[\tilde{u}|s]=q+\beta s$, which will then yield $E[\tilde{u} |s \leq a]= q+\beta E[s|s \leq a]$. I think that keeping the referenced post in mind and that $-\frac{f(a)}{F(a)}=-\sigma \frac{\phi(a)}{\Phi(a)}$ in order to get the final answer we need $\beta=\sigma$, but it is not. $\endgroup$ Commented Aug 22, 2023 at 17:53
  • $\begingroup$ It comes down to correctly relating $f(a)$ to $\phi(a).$ A common mistake is to forget the Jacobian. The use of the units calculus helps us remember to include it. For further explanation of using the units calculus in this situation, please see stats.stackexchange.com/a/624633/919. $\endgroup$
    – whuber
    Commented Aug 22, 2023 at 18:30
  • $\begingroup$ This is eye opening. I had never heard of 'units calculus' until today but I think after reading your post, the idea is that once you revert the temporarily employed unit variance back to its original variance (i.e., var(s) no longer equal to one), it cancels out the jacobian portion (i.e., the var of s) of the beta coefficient, which results in $\sigma^2$. However, I am not sure that $\beta Var(\tilde{u})$ is also equal to $\sigma^2$ $\endgroup$ Commented Aug 22, 2023 at 20:56
  • $\begingroup$ The latter is a basic covariance calculation. It uses the usual formula for $\beta$ in least squares regression (as a covariance divided by a variance) and computes both the variance and covariance from your definitions of $\tilde u$ and $s.$ Specifically, $$\operatorname{Var}(s) = \operatorname{Var}(\tilde u + \tilde e) = \operatorname{Var}(\tilde u) + \operatorname{Var}(\tilde e)$$ and $$\operatorname{Cov}(\tilde u, s) = \operatorname{Cov}(\tilde u, \tilde u + \tilde e)=\operatorname{Var}(\tilde u) + \operatorname{Cov}(\tilde u, \tilde e) = \operatorname{Var}(\tilde u).$$ $\endgroup$
    – whuber
    Commented Aug 22, 2023 at 22:02
  • $\begingroup$ That makes sense for $\beta Var(\tilde{s})=\sigma^2$ but was not sure how $\beta Var(\tilde{u})$ is also equal to $\sigma^2$ as written in your answer. $\endgroup$ Commented Aug 22, 2023 at 22:26
2
$\begingroup$

Conditional mean of normal variable, given the convolution

To answer the question in a bit more general terms, I use different notation here. $\newcommand{\E}{\mathbb{E}}\newcommand{\1}{\mathbb{1}}$ Let $X\sim N(\mu,\sigma^2_X$), $E\sim N(0,\sigma^2_E$) be independent. We observe the convolution $Y=X+E$ and are interested in the conditional mean $\E (X\mid Y\leq a)$.

Let $f_X, f_Y, f_E$ denote the pdfs of $X, Y, E$, and $F_X, F_Y, F_E$ the corresponding cdfs. The joint distribution of $X$ and $Y$ has pdf $$f_{X,Y}(x,y) = f_X(x)f_E(y-x),$$ and we can express the required conditional mean as $$ \begin{aligned} \E(X\mid Y\leq a)&=\frac{\int\int x f_{X,Y}(x,y)\1(y\leq a)dy\, dx}{\int\int f_{X,Y}(x,y)\1(y\leq a)dy\, dx} \\&= \frac{1}{F_Y(a)}\int\int x f_X(x)f_E(y-x)\1(y-x\leq a-x) dy\, dx \\&= \frac{1}{F_Y(a)}\int\int x f_X(x)\1(e\leq a-x) f_E(e) de\, dx \\&= \frac{1}{F_Y(a)}\int\underbrace{\left[\int x f_X(x)\1(x\leq a-e) dx\right]}_{\E(X\mid X\leq a-e)\cdot P(X\leq a-e)} f_E(e) de. \end{aligned} $$ Now we can apply the result in @whuber's answer from the thread linked in the question, replacing the standard normal pdf / density by $f_X(x)=\frac{1}{\sigma}\phi((x-\mu)/\sigma)$: $$ \E(X\mid X\leq a-e) = \mu-\sigma_X\frac{\sigma_X f_X(a-e)}{F_X(a-e)} = \mu-\sigma_X^2\frac{f_X(a-e)}{P(X\leq a-e)}. $$ This result uses the fact that $X$ is normal distributed. We get that $$ \begin{aligned} \E(X\mid Y\leq a)&= \frac{1}{F_Y(a)}\int{\E(X\mid X\leq a-e)\cdot P(X\leq a-e)} f_E(e) de \\&= \frac{1}{F_Y(a)}\int{\mu P(X\leq a-e) - \sigma_X^2 f_X(a-e)} f_E(e) de \\&= \frac{1}{F_Y(a)}\left[\mu \underbrace{\int P(X+e\leq a)f_E(e) de}_{P(Y \leq a)} - \sigma_X^2 \underbrace{\int f_X(a-e) f_E(e) de}_{f_Y(a)}\right] \\&= \mu - \sigma_X^2 \frac{f_Y(a)}{F_Y(a)}. \end{aligned} $$ This looks familiar, doesn't it? It is tempting to interpret this result as $\E (Y\mid Y\leq a)$. However, note that $Y\sim N(\mu, \sigma_X^2+\sigma_E^2)$, so $\sigma_Y^2= \sigma_X^2+\sigma_E^2$, and $$ \E (Y\mid Y\leq a) = \mu - \sigma_Y^2 \frac{f_Y(a)}{F_Y(a)} = \E(X\mid Y\leq a) -\sigma^2_E\frac{f_Y(a)}{F_Y(a)} . $$ The larger the noise variance $\sigma^2_E$, the bigger gets the discrepancy between $\E (Y\mid Y\leq a)$ and $\E (X\mid Y\leq a)$ - which makes sense: for $\sigma^2_E$, we have $Y=X$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.