Let $X$ and $Y$ be jointly distributed as a multivariate normal with the following parameters:

$$ \mu_{XY} = \begin{bmatrix} 0 \\ 0.2 \end{bmatrix} \qquad \Sigma_{XY} = \begin{bmatrix} 1 & 0.05 \\ 0.05 & 0.16 \end{bmatrix} $$

Now assume a third random variable, $Z = f(Y)$, created with the intention of discretizing $Y$. Specifically, we have

$$ Z = \begin{cases} 1 &y < F^{-1}(0.2)\\ 0 &\mathrm{otherwise} \end{cases}, $$ where $F^{-1}$ is the quantile function for $Y$, which means $F^{-1}(0.2)\approx -0.136$.

So I assume $Z \sim Bernoulli(0.2)$, which gives it the same mean and variance as $Y$.

My question is: how should the covariance matrix between $X$, $Y$ and $Z$ look? This is what I can immediately fill in:

$$ \Sigma_{XYZ} = \begin{bmatrix} 1 & 0.05 & ? \\ 0.05 & 0.16 & ? \\ ? & ? & 0.16\\ \end{bmatrix} $$

Which means that I need to figure out the covariances involving $Z$. My first instinct was that $\sigma_{XZ} = \sigma_{XY} = 0.05$ and $\sigma_{YZ} = 1$, but some simulated data showed this is not true. I can't get an analytic result either.

  • $\begingroup$ Yes, what I need in the end is a "true" covariance matrix, so I can do two things: 1) when I generate data from these three r.v., I want to compare the observed covariance matrix with the true one. 2) I want to compare linear regression coefficients generated from the data with some references, which I would retrieve from this "true" covariance matrix. $\endgroup$ – Waldir Leoncio Dec 31 '18 at 16:21
  • $\begingroup$ @JesperHybel, yes, right again! I got the categories flipped. I'll fix it right away. I had them right on paper and on code, though, so I'm still stuck. $\endgroup$ – Waldir Leoncio Dec 31 '18 at 16:38
  • $\begingroup$ The covariance $cov(Z,Y) = \mathbb E[ZY] - p \mathbb E[Y]$. And $\mathbb E[ZY] = \mathbb E[ZY\lvert Z=1] p(Z=1) + \mathbb E[ZY\lvert Z=0] p(Z=0)$ which reduce to $\mathbb E[Y\lvert Z=1] p(Z=1) = \mathbb E[Y \lvert Y < F_Y^{-1}(0.2)] p(Z=1)$. This is the expectation of a truncated normal link. No closed form solution, but can be expressed using cdf and pdf of normal. $\endgroup$ – Jesper Hybel Dec 31 '18 at 17:14

You could perhaps use the following

$Cov(Z,Y) = \mathbb E[ZY] - Pr(Z=1) \mathbb E[Y]$

found simply by applying the definition of covariance. Focusing on the term $\mathbb E[ZY]$ it then follows by total law of expectation that

$$ \mathbb E[ZY] = \mathbb E[ZY\lvert Z=1]Pr(Z=1) + \underbrace{\mathbb E[ZY\lvert Z=0]}_{=0}Pr(Z=0)$$

where one summand is seen to be $0$ such that the indentity becomes

$$ \mathbb E[ZY] = \mathbb E[ZY\lvert Z=1]Pr(Z=1) \\ = \mathbb E[Y\lvert Y < F_Y^{-1}(0.2)]Pr(Z=1) $$

This is the expectation of a truncated normal. No closed form solution, but can be expressed using cdf and pdf of normal.

Same logic should be applicable to $Cov(Z,X)$ to get

$Cov(Z,X) = \mathbb E[X\lvert Y < F_Y^{-1}(0.2)]Pr(Z=1) - Pr(Z=1) \mathbb E[X]$

and then you can look here link for an expression of the expectation $\mathbb E[X\lvert Y < F_Y^{-1}(0.2)]$ which is the only part assumed not given in the problem.

  • 1
    $\begingroup$ Thank you for taking the time to write this. The Math checks out, but I am still working it out in practice. Happy 2019! $\endgroup$ – Waldir Leoncio Jan 1 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.