1
$\begingroup$

Let $X_1,...,X_{n_1}$ be an i.i.d. sample from $N_p(\mu_1,\Sigma)$ and let $Y_1,...,Y_{n_2}$ be an independent sample from $N_p(\mu_2,\Sigma)$, for some $\mu_1,\mu_2 \in \mathbb{R}^p$ and some invertible, $p\times p$ positive definite matrix $\Sigma$.

I'd like to find the likelihood function $L(\mu_1,\mu_2,\Sigma)$ of the commbined sample:

In my book, the likelihood function of $X_1,...,X_n \sim N_p(\mu,\Sigma)$ is given by

$$\frac{1}{(2\pi)^{np/2}\text{det}(\Sigma)^{n/2}}\exp\biggl(-1/2\bigl(\sum^n_{i=1}(x_i-\mu)^T\Sigma^{-1}(x_i-\mu)\bigr)\biggr)$$

So, $$L(\mu_1,\mu_2,\Sigma)=$$ $$\frac{1}{(2\pi)^{p(n_1+n_2)/2}\text{det}(\Sigma)^{\frac{n_1+n_2}{2}}}\exp\biggl(-1/2\sum^{n_1}_{i=1}(x_i-\mu_1)^T\Sigma^{-1}(x_i-\mu_1)-1/2\sum^{n_2}_{i=1}(y_i-\mu_2)^T\Sigma^{-1}(y_i-\mu_2)\biggr)$$

So I would like to find the MLE of $\Sigma$:

$$\mathcal{L}(\mu_1,\mu_2,\Sigma)\propto\det(\Sigma)^{\frac{n_1+n_2}{2}}\exp\biggl(-\frac{1}{2}\bigl(\sum^{n_1}_{i=1}(x_i-\mu_1)^T\Sigma^{-1}(x_i-\mu_1)+\sum^{n_2}_{j=1}(y_j-\mu_2)^T\Sigma^{-1}(y_j-\mu_2)\bigr)\biggr)$$

We can say that $(x_i-\mu_1)^T\Sigma^{-1}(x_i-\mu_1)+(y_j-\mu_2)^T\Sigma^{-1}(y_j-\mu_2)$ is the trace of a $1\times 1$ matrix, so

$$=\det(\Sigma)^{\frac{n_1+n_2}{2}}\exp\biggl(-\frac{1}{2}\bigl(\sum^{n_1}_{i=1}Tr((x_i-\mu_1)(x_i-\mu_1)^T\Sigma^{-1})+\sum^{n_2}_{j=1}Tr((y_j-\mu_2)(y_j-\mu_2)^T\Sigma^{-1}\bigr)\biggr)$$

$$=\det(\Sigma)^{\frac{n_1+n_2}{2}}\exp\biggl(-\frac{1}{2}\bigl(Tr(S_x\Sigma^{-1})+Tr(S_y\Sigma^{-1})\bigr)\biggr)$$

where $S_x=\sum^{n_1}_{i=1}(x_i-\mu_1)(x_i-\mu_1)^T\in \mathbb{R}^{p\times p}$

$S_y=\sum^{n_2}_{j=1}(y_j-\mu_2)(y_j-\mu_2)^T\in \mathbb{R}^{p\times p}$

and with $S=S^{1/2}S^{1/2}$:

$$=\det(\Sigma)^{\frac{n_1+n_2}{2}}\exp(-\frac{1}{2}\biggl(Tr\bigl(S^{1/2}_x\Sigma^{-1}S^{1/2}_x\bigr)+Tr\bigl(S^{1/2}_y\Sigma^{-1}S_y^{1/2}\bigr)\biggr)$$

Let $B_x=S_x^{1/2}\Sigma^{-1}S_x^{1/2},B_y=S_y\Sigma^{-1}S_y^{1/2}$

So,

$$=\det(S_x)^{\frac{-n_1}{2}}\det(B_x)^{\frac{n_1}{2}}\det(S_y)^{\frac{-n_2}{2}}\det(B_y)^{\frac{n_2}{2}}\exp\biggl(\frac{-1}{2}Tr(B_x+B_y)\biggr)$$

and now i'm stuck. What i'm trying to do is on this wikipedia page, but i'm struggling to use it on my example. Any help would be highly appreciated.

$\endgroup$
0

1 Answer 1

0
$\begingroup$

$S^*=S_x+S_y$ should take the role of $S$ in the Wikipedia proof. Use $Tr(S_x\Sigma^{-1})+Tr(S_y\Sigma^{-1})=Tr(S^* \Sigma^{-1})$, then you should be able to go on as on Wikipedia.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.