# Deriving the unconditional covariance matrix of the mean vector

Let $y\sim Bin(n,0.5)$ and $X|y\sim N_p(\mu_y,\Sigma_y)$, where $\mu_0,\mu_1\in\mathbb{R}^p$ and $\Sigma_0,\Sigma_1\in\mathbb{R}^{p\times p}$.

Knowing that $\bar{X}\sim N_p(\mu_x,\frac{1}{n}\Sigma_x)$, all I need is to find the unconditional distribution of X. Starting with $E[X]$: $$E[X]=E[E[X|y]]=E[X|y=0]P(y=0)+E[X|y=1]P(y=1)=0.5\mu_0+0.5\mu_1$$ The problem is with the unconditional covariance - I want to use the total variance law: $$Cov(X)=E[Cov(X|y)]+Cov(E[X|y])$$ While the left term is easily found: $$E[Cov(X|y)]=Cov(X|y=0)P(y=0)+Cov(X|y=1)P(y=1)=0.5\Sigma_0+0.5\Sigma_1$$ I keep on messing with the right one (which should also be a $p\times p$ matrix: $Cov(z)=E[zz^T]-E[z]E[z]^T$, but then again I need to define $E[X|y]$ somehow, my best idea is $E[X|y]=0.5\mu_0+0.5\mu_1$ but this feels like the wrong answer.

Any ideas?

• +1, I am not sure on this. Since you already know the distribution of $X|y\sim N_p(\mu_y,\Sigma_y)$ , does this mean $E[X|y]=\mu_y$ which can be treated as constant, then $Cov(E[X|y])=0?$ Nov 6, 2017 at 12:07
• @DeepNorth Careful: $\mu_y$ is not constant. It is a function of $y$. The notation is a little confusing here. Nov 6, 2017 at 12:24
• Yes, you are right $E[X|y]$should be of funciton of y Nov 6, 2017 at 12:34
• I think it's all down to notation. $\mu_y$ is constant for a given $y$, but here $y$ is actually a random variable. I think it's better to use $Y$ for the r.v. and $y$ for particular values. Nov 6, 2017 at 12:37

I would recommend using the conventional notation for random variables.

$E[X|Y]$ is actually a function of $Y$, and is a random variable itself.

In your case, observe that $E[X|Y=y]$ is simply $\mu_y$.

Thus,

$$Cov(E[X|Y]) = \frac{1}{2}(\mu_0\mu_0^T+\mu_1\mu_1^T)-\frac{1}{2}(\mu_0+\mu_1)\frac{1}{2}(\mu_0+\mu_1)^T$$

Also, did you mean Bernoulli instead of Binomial?

• 1. This is the answer I've come with as well, thanks. 2. No, Binomial on purpose. After finding the covariance of X, it is easy to derive the mean's covariance as n samples are iid and thus $Cov(\bar{X})=\frac{1}{n}Cov(X)$ Nov 6, 2017 at 12:59