3
$\begingroup$

Let $y\sim Bin(n,0.5)$ and $X|y\sim N_p(\mu_y,\Sigma_y)$, where $\mu_0,\mu_1\in\mathbb{R}^p$ and $\Sigma_0,\Sigma_1\in\mathbb{R}^{p\times p}$.

Knowing that $\bar{X}\sim N_p(\mu_x,\frac{1}{n}\Sigma_x)$, all I need is to find the unconditional distribution of X. Starting with $E[X]$: $$E[X]=E[E[X|y]]=E[X|y=0]P(y=0)+E[X|y=1]P(y=1)=0.5\mu_0+0.5\mu_1$$ The problem is with the unconditional covariance - I want to use the total variance law: $$Cov(X)=E[Cov(X|y)]+Cov(E[X|y])$$ While the left term is easily found: $$E[Cov(X|y)]=Cov(X|y=0)P(y=0)+Cov(X|y=1)P(y=1)=0.5\Sigma_0+0.5\Sigma_1$$ I keep on messing with the right one (which should also be a $p\times p$ matrix: $Cov(z)=E[zz^T]-E[z]E[z]^T$, but then again I need to define $E[X|y]$ somehow, my best idea is $E[X|y]=0.5\mu_0+0.5\mu_1$ but this feels like the wrong answer.

Any ideas?

$\endgroup$
4
  • $\begingroup$ +1, I am not sure on this. Since you already know the distribution of $X|y\sim N_p(\mu_y,\Sigma_y)$ , does this mean $E[X|y]=\mu_y$ which can be treated as constant, then $Cov(E[X|y])=0?$ $\endgroup$
    – Deep North
    Nov 6, 2017 at 12:07
  • 1
    $\begingroup$ @DeepNorth Careful: $\mu_y$ is not constant. It is a function of $y$. The notation is a little confusing here. $\endgroup$
    – cangrejo
    Nov 6, 2017 at 12:24
  • $\begingroup$ Yes, you are right $E[X|y] $should be of funciton of y $\endgroup$
    – Deep North
    Nov 6, 2017 at 12:34
  • $\begingroup$ I think it's all down to notation. $\mu_y$ is constant for a given $y$, but here $y$ is actually a random variable. I think it's better to use $Y$ for the r.v. and $y$ for particular values. $\endgroup$
    – cangrejo
    Nov 6, 2017 at 12:37

1 Answer 1

2
$\begingroup$

I would recommend using the conventional notation for random variables.

$E[X|Y]$ is actually a function of $Y$, and is a random variable itself.

In your case, observe that $E[X|Y=y]$ is simply $\mu_y$.

Thus,

$$ Cov(E[X|Y]) = \frac{1}{2}(\mu_0\mu_0^T+\mu_1\mu_1^T)-\frac{1}{2}(\mu_0+\mu_1)\frac{1}{2}(\mu_0+\mu_1)^T $$

Also, did you mean Bernoulli instead of Binomial?

$\endgroup$
1
  • $\begingroup$ 1. This is the answer I've come with as well, thanks. 2. No, Binomial on purpose. After finding the covariance of X, it is easy to derive the mean's covariance as n samples are iid and thus $Cov(\bar{X})=\frac{1}{n}Cov(X)$ $\endgroup$
    – Spätzle
    Nov 6, 2017 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.