# How to calculate the covariance matrix for a categorized variable?

Let $$X$$ and $$Y$$ be jointly distributed as a multivariate normal with the following parameters:

$$\mu_{XY} = \begin{bmatrix} 0 \\ 0.2 \end{bmatrix} \qquad \Sigma_{XY} = \begin{bmatrix} 1 & 0.05 \\ 0.05 & 0.16 \end{bmatrix}$$

Now assume a third random variable, $$Z = f(Y)$$, created with the intention of discretizing $$Y$$. Specifically, we have

$$Z = \begin{cases} 1 &y < F^{-1}(0.2)\\ 0 &\mathrm{otherwise} \end{cases},$$ where $$F^{-1}$$ is the quantile function for $$Y$$, which means $$F^{-1}(0.2)\approx -0.136$$.

So I assume $$Z \sim Bernoulli(0.2)$$, which gives it the same mean and variance as $$Y$$.

My question is: how should the covariance matrix between $$X$$, $$Y$$ and $$Z$$ look? This is what I can immediately fill in:

$$\Sigma_{XYZ} = \begin{bmatrix} 1 & 0.05 & ? \\ 0.05 & 0.16 & ? \\ ? & ? & 0.16\\ \end{bmatrix}$$

Which means that I need to figure out the covariances involving $$Z$$. My first instinct was that $$\sigma_{XZ} = \sigma_{XY} = 0.05$$ and $$\sigma_{YZ} = 1$$, but some simulated data showed this is not true. I can't get an analytic result either.

• Yes, what I need in the end is a "true" covariance matrix, so I can do two things: 1) when I generate data from these three r.v., I want to compare the observed covariance matrix with the true one. 2) I want to compare linear regression coefficients generated from the data with some references, which I would retrieve from this "true" covariance matrix. – Waldir Leoncio Dec 31 '18 at 16:21
• @JesperHybel, yes, right again! I got the categories flipped. I'll fix it right away. I had them right on paper and on code, though, so I'm still stuck. – Waldir Leoncio Dec 31 '18 at 16:38
• The covariance $cov(Z,Y) = \mathbb E[ZY] - p \mathbb E[Y]$. And $\mathbb E[ZY] = \mathbb E[ZY\lvert Z=1] p(Z=1) + \mathbb E[ZY\lvert Z=0] p(Z=0)$ which reduce to $\mathbb E[Y\lvert Z=1] p(Z=1) = \mathbb E[Y \lvert Y < F_Y^{-1}(0.2)] p(Z=1)$. This is the expectation of a truncated normal link. No closed form solution, but can be expressed using cdf and pdf of normal. – Jesper Hybel Dec 31 '18 at 17:14

You could perhaps use the following

$$Cov(Z,Y) = \mathbb E[ZY] - Pr(Z=1) \mathbb E[Y]$$

found simply by applying the definition of covariance. Focusing on the term $$\mathbb E[ZY]$$ it then follows by total law of expectation that

$$\mathbb E[ZY] = \mathbb E[ZY\lvert Z=1]Pr(Z=1) + \underbrace{\mathbb E[ZY\lvert Z=0]}_{=0}Pr(Z=0)$$

where one summand is seen to be $$0$$ such that the indentity becomes

$$\mathbb E[ZY] = \mathbb E[ZY\lvert Z=1]Pr(Z=1) \\ = \mathbb E[Y\lvert Y < F_Y^{-1}(0.2)]Pr(Z=1)$$

This is the expectation of a truncated normal. No closed form solution, but can be expressed using cdf and pdf of normal.

Same logic should be applicable to $$Cov(Z,X)$$ to get

$$Cov(Z,X) = \mathbb E[X\lvert Y < F_Y^{-1}(0.2)]Pr(Z=1) - Pr(Z=1) \mathbb E[X]$$

and then you can look here link for an expression of the expectation $$\mathbb E[X\lvert Y < F_Y^{-1}(0.2)]$$ which is the only part assumed not given in the problem.

• Thank you for taking the time to write this. The Math checks out, but I am still working it out in practice. Happy 2019! – Waldir Leoncio Jan 1 at 10:05