Conditional Expected Value of Product of Normal and Log-Normal Distribution

Question

Could someone please provide the answer and steps to solve this expression?

\begin{eqnarray*} E\left[\left.\left(e^{X}Y+k\right)\right|\left.\left(e^{X}Y+k\right)>0\right]\right. \end{eqnarray*} $E$ is the expectation operator. \begin{eqnarray*} X\sim N\left(\mu_{X},\sigma_{X}^{2}\right);Y\sim N\left(\mu_{Y},\sigma_{Y}^{2}\right);X\;\text{and }Y\text{ are independent. Also, }k<0 \end{eqnarray*}

KEY MISSING LINK

The above expression depends on proving the below general expression or at least showing that it holds for the special case of Normal and Log-Normal distribution.

$E[UV\mid UV>c] = E[UE[V\mid UV>c]]$

Here, $U,V$ are independent random variables that could be discrete or continuos and follow any probability distribution. $E$ is the expectation operator. $c$ is a constant.

Of course, for our main question we require only the case where one of them is normally distributed and the other is log-normal? Say, $U$ is log-normal and $V$ is normal. Is the above identity true for this special case?

This is posed as a separate question due to its importance: Proof of Simplification of Conditional Expectation of Product of Random Variables

STEPS TRIED

0) JOINT CONDITIONAL DENSITY

I am having difficulty coming up with the conditional joint density to use in the above expectation. The joint density function of just $X$ and $Y$ is straight forward and follows from the standard density function for the bivariate normal case. How would we incorporate the conditional aspect into the joint density function?

1) NORMAL LOG-NORMAL MIXTURE PAPER BY YANG

(Link: http://repec.org/esAUSM04/up.21034.1077779387.pdf)

This paper has the first four central moments without proof (Equation 5 in above paper). If someone could provide these proofs, that might shed more light on the problem above.

The variables in the Yang paper are correlated, which is easy to apply to above; but they also have zero mean in the paper, which in our case does not apply directly, since we have non zero mean.

2)OTHER RELATED LINKS

a) Interesting question about an expectation involving a slightly modified form of the normal log-normal mixture. Though this lacks the conditional aspect, and hence needs some modification before it can be used for the problem above.

https://math.stackexchange.com/questions/1142841/covariance-in-normal-lognormal-nln-mixture

b) Another question on the normal log-normal mixture though this lacks a deeper discussion.

https://math.stackexchange.com/questions/159818/combination-of-a-normal-r-v-with-a-log-normal-one

c) Question on conditional expectation of product of independent random variables. It would be good to know which aspects from this are applicable in our case.

https://math.stackexchange.com/questions/544410/result-and-proof-on-the-conditional-expectation-of-the-product-of-two-random-var

d) Other interesting questions on conditional expectation of independent random variables.

https://math.stackexchange.com/questions/380866/conditional-expectations-for-independent-random-variables?rq=1

https://math.stackexchange.com/questions/55524/rule-with-independent-random-variables-and-conditional-expectations?rq=1

3) TAYLOR SERIES APPROXIMATIONS

Would it be possible to use taylor serious approximations here? I am little confused due to the conditional expectation and the normal log normal mixture? Any pointers on whether this is possible and how to proceed further or whether this is not applicable here would be great.

4) USING STANDARD NORMAL (SEEMS LIKE A DEAD END)

I know that if we can express this sum using the standard normal as below, there is a solution. Please advice on how to do this or other alternatives to solve the above would be helpful as well. This seems to be a dead end as confirmed by experts on this forum. But still keeping here if someone discovers a way to continue using this approach.

\begin{eqnarray*} W=\left(e^{X}Y+k\right)<=>\mu+\sigma Z\text{ where, }Z\sim N\left(0,1\right) \end{eqnarray*}

\begin{eqnarray*} \left[W\sim N\left(\mu,\sigma^{2}\right)\Rightarrow W=\mu+\sigma Z\;;\; W>0\Rightarrow Z>-\mu/\sigma\right] \end{eqnarray*} We then need to determine, $\mu\text{ and }\sigma$.

We have for every standard normal distribution, $Z$, and for every $u,$ $Pr\left[Z>\text-u\right]=Pr\left[Z<u\right]=\mathbf{\Phi}\left(u\right)$. Here, $\phi$ and $\mathbf{\Phi}$ are the standard normal PDF and CDF, respectively. \begin{eqnarray*} E\left[\left.Z\right|Z>-u\right] & = & \frac{1}{\mathbf{\Phi}\left(u\right)}\left[\int_{-u}^{\infty}t\phi\left(t\right)dt\right]\\ & = & \frac{1}{\mathbf{\Phi}\left(u\right)}\left[\left.-\phi\left(t\right)\right|_{-u}^{\infty}\right]=\frac{\phi\left(u\right)}{\mathbf{\Phi}\left(u\right)} \end{eqnarray*} Hence we have, \begin{eqnarray*} E\left[\left.Y\right|Y>0\right] & = & \mu+\sigma E\left[\left.Z\right|Z>\left(-\frac{\mu}{\sigma}\right)\right]\\ & = & \mu+\frac{\sigma\phi\left(\mu/\sigma\right)}{\mathbf{\Phi}\left(\mu/\sigma\right)} \end{eqnarray*} Setting, $\psi\left(u\right)=u+\phi\left(u\right)/\Phi\left(u\right)$, \begin{eqnarray*} E\left[\left.Y\right|Y>0\right]=\sigma\psi\left(\mu/\sigma\right) \end{eqnarray*}

Answer 1

What is the intended use of the result? That bears on what form of answer is needed, to include whether a stochastic (Monte Carlo) simulation approach might be adequate, And even the bigger picture matter of is this problem necessary to solve, and did someone come up with this problem as a way of solving a higher level problem, and there might be a better approach to the higher level problem which doesn't require this.

Here is a stochastic (Monte Carlo) simulation solution in MATLAB.

a = 1; b = 2; c = 3; d = 4; k = -1; % Made up values for illustrative purpose
n = 1e8; % Number of replications
mux = 10; sigmax = 4; sigmay = 7; % Made up values for illustrative purposes
X = mux + sigmax * randn(n,1); Y = sigmay * randn(n,1); Y1 = a + b + c + d * Y;
success_index = exp(X).*Y1 > 0; % replications in which condition is true
num_success = sum(success_index);
Cond_Sample = exp(X(success_index)) .* Y1(success_index) + k;
disp([num_success mean(Cond_Sample) std(Cond_Sample)/sqrt(num_success)])
1.0e+09 *
0.058475265000000   1.502775087443930   0.057342191058931

Answer 2

Comments:

1. The joint density is given by multiplying the densities since they are idp. One variable is just a paramater to the other.

2. $Ye^X$ is not normally distributed so approach (4) wont work.

3. The expressions below might allow you to find some approximation. If not they are relatively easy to evaluate with a computer.

Let $X\sim\mathcal{N}(\mu, \sigma^2)$, denote $c = -k/y$, and $d(y)=\log(-k/y)$. Let $Z=e^X$, then $$E[YZ\mid B] = E[YE[Z\mid B]],$$ and, $$E[Z\mid Z>c] = e^{\mu+\sigma^2/2}\frac{P(X>d-\sigma^2)}{P(X>d)},$$ if $c>0$ and $E[Z] = e^{\mu+\sigma^2/2}$ otherwise. Thus, since $c>0$ only if $Y<0$, $$E[YZ\mid B] = \frac{1}{2}e^{\mu+\sigma^2/2}\bigg(E[Y\mathbb{1}(Y>0] + E\Big[Y\frac{P(X>d(Y)-\sigma^2)}{P(X>d(Y))}\mathbb{1}(Y<0)\Big]\bigg).$$

The first part is simply $$\int_0^\infty y f_Y(y)\,dy,$$ and the second $$\int_{-\infty}^0 y \frac{P(X>log(-\frac{k}{y})-\sigma^2)}{P(X>log(-\frac{k}{y}))}f_Y(y)\,dy.$$

Answer 3

This solution is due to the suggestions and corrections from @Hunaphu, @whuber, and others. Could someone please verify if all the steps make sense?

ANSWER STEPS START

Using some notational shortcuts,

Consider, \begin{eqnarray*} E\left[\left.\left(e^{X}Y+k\right)\right|\left(e^{X}Y+k\right)>0\right] & = & E\left[k\left|\left(e^{X}Y+k\right)>0\right.\right]+E\left[\left(e^{X}Y\right)\left|\left(e^{X}Y+k\right)>0\right.\right] \end{eqnarray*} \begin{eqnarray*} & = & k+E\left[\left.\left(Ye^{X}\right)\right|\left.\left(Ye^{X}+k\right)>0\right]\right. \end{eqnarray*} \begin{eqnarray*} & = & k+\int\int ye^{x}f\left(\left.ye^{x}\right|\left\{ ye^{x}+k\right\} >0\right)dxdy \end{eqnarray*} Here, $f\left(w\right)$ is the probability density function for $w$, Here, $f\left(w\right)$ is the probability density function for $w$, \begin{eqnarray*} & = & k+\int\int ye^{x}\frac{f\left(ye^{x};\left\{ ye^{x}+k\right\} >0\right)}{f\left(\left\{ ye^{x}+k\right\} >0\right)}dxdy \end{eqnarray*} \begin{eqnarray*} \left[\text{We note that, }ye^{x}>-k>0\Rightarrow y>0\right] \end{eqnarray*} \begin{eqnarray*} & = & k+\int\int ye^{x}\frac{f\left(y\right)f\left(e^{x};\left\{ ye^{x}+k\right\} >0\right)}{f\left(\left\{ ye^{x}+k\right\} >0\right)}dxdy \end{eqnarray*} \begin{eqnarray*} & = & k+\int y\left[\int\frac{e^{x}f\left(e^{x};\left\{ e^{x}>-\frac{k}{y}\right\} \right)}{f\left(e^{x}>-\frac{k}{y}\right)}dx\right]f\left(y\right)dy \end{eqnarray*} \begin{eqnarray*} & = & k+\int y\left[\int e^{x}f\left(e^{x}\left|\left\{ e^{x}>-\frac{k}{y}\right\} \right.\right)dx\right]f\left(y\right)dy \end{eqnarray*} \begin{eqnarray*} & = & k+\int_{0}^{\left(y<-k\right)}y\left[\int e^{x}f\left(e^{x}\left|\left\{ e^{x}>1\right\} \right.\right)dx\right]f\left(y\right)dy+\int_{\left(y>-k\right)}^{\infty}y\left[\int e^{x}f\left(e^{x}\left|\left\{ e^{x}<1\right\} \right.\right)dx\right]f\left(y\right)dy \end{eqnarray*} \begin{eqnarray*} & = & k+\int_{0}^{\left(-k\right)}y\left[E\left(\left.W\right|W>c\right)\right]f\left(y\right)dy+\int_{\left(-k\right)}^{\infty}y\left[E\left(\left.W\right|W<c\right)\right]f\left(y\right)dy\quad;\;\text{here, }W=e^{X}\text{ and }c=1 \end{eqnarray*} Simplifying the inner expectations, \begin{eqnarray*} E\left(\left.W\right|W>c\right)=\frac{1}{P\left(e^{X}>c\right)}\int_{c}^{\infty}w\frac{1}{w\sigma_{X}\sqrt{2\pi}}e^{-\frac{1}{2}\left[\frac{ln\left(w\right)-\mu_{X}}{\sigma_{X}}\right]^{2}}dw \end{eqnarray*} Put $t=ln\left(w\right)$, we have, $dw=e^{t}dt$ \begin{eqnarray*} E\left(\left.W\right|W>c\right)=\frac{1}{P\left(X>ln\left(c\right)\right)}\int_{ln\left(c\right)}^{\infty}\frac{e^{t}}{\sigma_{X}\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{t-\mu_{X}}{\sigma_{X}}\right)^{2}}dt \end{eqnarray*} \begin{eqnarray*} t-\frac{1}{2}\left(\frac{t-\mu_{X}}{\sigma_{X}}\right)^{2}=-\frac{1}{2\sigma_{X}^{2}}\left(t-\left(\mu_{X}+\sigma_{X}^{2}\right)\right)^{2}+\mu_{X}+\frac{\sigma_{X}^{2}}{2} \end{eqnarray*} \begin{eqnarray*} E\left(\left.W\right|W>c\right)=\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{P\left(\mu_{X}+\sigma_{X}Z>ln\left(c\right)\right)}\int_{ln\left(c\right)}^{\infty}\frac{1}{\sigma_{X}\sqrt{2\pi}}e^{-\frac{1}{2}\left[\frac{t-\left(\mu_{X}+\sigma_{X}^{2}\right)}{\sigma_{X}}\right]^{2}}dt\quad;Z\sim N\left(0,1\right) \end{eqnarray*} Put $s=\left[\frac{t-\left(\mu_{X}+\sigma_{X}^{2}\right)}{\sigma_{X}}\right]$ and $b=\left[\frac{ln\left(c\right)-\left(\mu_{X}+\sigma_{X}^{2}\right)}{\sigma_{X}}\right]$ we have, $ds=\frac{dt}{\sigma_{X}}$ \begin{eqnarray*} E\left(\left.W\right|W>c\right)=\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{P\left(Z>\frac{ln\left(c\right)-\mu_{X}}{\sigma_{X}}\right)}\int_{b}^{\infty}\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}s^{2}}ds \end{eqnarray*} \begin{eqnarray*} =\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{P\left(Z<\frac{-ln\left(c\right)+\mu_{X}}{\sigma_{X}}\right)}\left[\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}s^{2}}ds-\int_{-\infty}^{b}\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}s^{2}}ds\right] \end{eqnarray*} \begin{eqnarray*} =\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{P\left(Z<\frac{-ln\left(c\right)+\mu_{X}}{\sigma_{X}}\right)}\left[1-\Phi\left(b\right)\right]\quad;\Phi\text{ is the standard normal CDF} \end{eqnarray*} \begin{eqnarray*} =\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{\Phi\left(\frac{-ln\left(c\right)+\mu_{X}}{\sigma_{X}}\right)}\left[\Phi\left(-b\right)\right] \end{eqnarray*} Similarly for the other case, \begin{eqnarray*} E\left(\left.W\right|W<c\right)=\frac{1}{P\left(e^{X}<c\right)}\int_{0}^{c}w\frac{1}{w\sigma_{X}\sqrt{2\pi}}e^{-\frac{1}{2}\left[\frac{ln\left(w\right)-\mu_{X}}{\sigma_{X}}\right]^{2}}dw \end{eqnarray*} Put $t=ln\left(w\right)$, we have, $dw=e^{t}dt$ \begin{eqnarray*} E\left(\left.W\right|W<c\right)=\frac{1}{P\left(X<ln\left(c\right)\right)}\int_{-\infty}^{ln\left(c\right)}\frac{e^{t}}{\sigma_{X}\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{t-\mu_{X}}{\sigma_{X}}\right)^{2}}dt \end{eqnarray*} \begin{eqnarray*} t-\frac{1}{2}\left(\frac{t-\mu_{X}}{\sigma_{X}}\right)^{2}=-\frac{1}{2\sigma_{X}^{2}}\left(t-\left(\mu_{X}+\sigma_{X}^{2}\right)\right)^{2}+\mu_{X}+\frac{\sigma_{X}^{2}}{2} \end{eqnarray*} \begin{eqnarray*} E\left(\left.W\right|W<c\right)=\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{P\left(\mu_{X}+\sigma_{X}Z<ln\left(c\right)\right)}\int_{-\infty}^{ln\left(c\right)}\frac{1}{\sigma_{X}\sqrt{2\pi}}e^{-\frac{1}{2}\left[\frac{t-\left(\mu_{X}+\sigma_{X}^{2}\right)}{\sigma_{X}}\right]^{2}}dt\quad;Z\sim N\left(0,1\right) \end{eqnarray*} Put $s=\left[\frac{t-\left(\mu_{X}+\sigma_{X}^{2}\right)}{\sigma_{X}}\right]$ and $b=\left[\frac{ln\left(c\right)-\left(\mu_{X}+\sigma_{X}^{2}\right)}{\sigma_{X}}\right]$ we have, $ds=\frac{dt}{\sigma_{X}}$ \begin{eqnarray*} E\left(\left.W\right|W<c\right)=\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{P\left(Z<\frac{ln\left(c\right)-\mu_{X}}{\sigma_{X}}\right)}\int_{-\infty}^{b}\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}s^{2}}ds \end{eqnarray*} \begin{eqnarray*} =\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{P\left(Z<\frac{ln\left(c\right)-\mu_{X}}{\sigma_{X}}\right)}\left[\Phi\left(b\right)\right]\quad;\Phi\text{ is the standard normal CDF} \end{eqnarray*} \begin{eqnarray*} =\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{\Phi\left(\frac{ln\left(c\right)-\mu_{X}}{\sigma_{X}}\right)}\left[\Phi\left(b\right)\right] \end{eqnarray*} Using the results for the inner expectations, \begin{eqnarray*} E\left[\left.\left(e^{X}Y+k\right)\right|\left(e^{X}Y+k\right)>0\right]=k+\int_{0}^{\left(-k\right)}y\left[\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{\Phi\left(\frac{-ln\left(c\right)+\mu_{X}}{\sigma_{X}}\right)}\left[\Phi\left(-b\right)\right]\right]f\left(y\right)dy+\int_{\left(-k\right)}^{\infty}y\left[\frac{e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}}{\Phi\left(\frac{ln\left(c\right)-\mu_{X}}{\sigma_{X}}\right)}\left[\Phi\left(b\right)\right]\right]f\left(y\right)dy \end{eqnarray*} \begin{eqnarray*} =k+e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}\left[\int_{0}^{\left(-k\right)}y\left\{ \frac{\Phi\left(\frac{\mu_{X}+\sigma_{X}^{2}}{\sigma_{X}}\right)}{\Phi\left(\frac{\mu_{X}}{\sigma_{X}}\right)}\right\} f\left(y\right)dy+\int_{\left(-k\right)}^{\infty}y\left\{ \frac{\Phi\left(-\left[\frac{\mu_{X}+\sigma_{X}^{2}}{\sigma_{X}}\right]\right)}{\Phi\left(-\left[\frac{\mu_{X}}{\sigma_{X}}\right]\right)}\right\} f\left(y\right)dy\right] \end{eqnarray*} \begin{eqnarray*} =k+e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}\left[\int_{0}^{\left(-k\right)}y\left\{ \frac{\Phi\left(\frac{\mu_{X}+\sigma_{X}^{2}}{\sigma_{X}}\right)}{\Phi\left(\frac{\mu_{X}}{\sigma_{X}}\right)}\right\} f\left(y\right)dy+\int_{\left(-k\right)}^{\infty}y\left\{ \frac{1-\Phi\left(\frac{\mu_{X}+\sigma_{X}^{2}}{\sigma_{X}}\right)}{1-\Phi\left(\frac{\mu_{X}}{\sigma_{X}}\right)}\right\} f\left(y\right)dy\right] \end{eqnarray*} \begin{eqnarray*} & = & k+e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}\left[\left\{ \frac{\Phi\left(\frac{\mu_{X}+\sigma_{X}^{2}}{\sigma_{X}}\right)}{\Phi\left(\frac{\mu_{X}}{\sigma_{X}}\right)}\right\} \int_{-\frac{\mu_{Y}}{\sigma_{Y}}}^{-\left(\frac{k+\mu_{Y}}{\sigma_{Y}}\right)}\left(\mu_{Y}+\sigma_{Y}z\right)\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}z^{2}}dz\right.\\ & & +\left.\left\{ \frac{1-\Phi\left(\frac{\mu_{X}+\sigma_{X}^{2}}{\sigma_{X}}\right)}{1-\Phi\left(\frac{\mu_{X}}{\sigma_{X}}\right)}\right\} \int_{-\left(\frac{k+\mu_{Y}}{\sigma_{Y}}\right)}^{\infty}\left(\mu_{Y}+\sigma_{Y}z\right)\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}z^{2}}dz\right]\quad;Z\sim N\left(0,1\right) \end{eqnarray*} \begin{eqnarray*} & = & k+e^{\left(\mu_{X}+\frac{1}{2}\sigma_{X}^{2}\right)}\left[\left\{ \frac{\Phi\left(\frac{\mu_{X}+\sigma_{X}^{2}}{\sigma_{X}}\right)}{\Phi\left(\frac{\mu_{X}}{\sigma_{X}}\right)}\right\} \left\{ \mu_{Y}\left[\Phi\left(-\left[\frac{k+\mu_{Y}}{\sigma_{Y}}\right]\right)-\Phi\left(-\frac{\mu_{Y}}{\sigma_{Y}}\right)\right]-\frac{\sigma_{Y}}{\sqrt{2\pi}}\left[e^{-\frac{1}{2}\left(\frac{k+\mu_{Y}}{\sigma_{Y}}\right)^{2}}-e^{-\frac{1}{2}\left(\frac{\mu_{Y}}{\sigma_{Y}}\right)^{2}}\right]\right\} \right.\\ & & +\left.\left\{ \frac{1-\Phi\left(\frac{\mu_{X}+\sigma_{X}^{2}}{\sigma_{X}}\right)}{1-\Phi\left(\frac{\mu_{X}}{\sigma_{X}}\right)}\right\} \left\{ \mu_{Y}\left[1-\Phi\left(-\left[\frac{k+\mu_{Y}}{\sigma_{Y}}\right]\right)\right]+\frac{\sigma_{Y}}{\sqrt{2\pi}}\left[e^{-\frac{1}{2}\left(\frac{k+\mu_{Y}}{\sigma_{Y}}\right)^{2}}\right]\right\} \right] \end{eqnarray*}