1
$\begingroup$

Suppose $X$ is a $p$-dimensional vector following $N_p(\mu,\Sigma)$ distribution, where $\mu$ is $p$-dimensional and $\Sigma$ is $p\times p$. Let $X=\left(\begin{array}{ccc}X_1\\X_2\end{array} \right)$ and $\Sigma=\left( \begin{array}{ccc} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21}&\Sigma_{22} \end{array} \right)$ where $X_1$ is $p_1\times1$ and $X_2$ is $p_2\times1$ with $p_1+p_2=p$. Also we have by usual notations $\Sigma_{12}=\text{Cov}(X_1,X_2)=\Sigma_{21}^T$. Show that $$Q:=X^T\Sigma^{-1}X-X_1^T\Sigma_{11}^{-1}X_1$$follows $\chi^2(p_2)$.

I tried to proceed making the changes $Y=\Sigma^{-1/2}X$ and $Z=\Sigma_{11}^{-1/2}X_1$ but am not really sure if it is in the right direction. Also, I tried to proceed by considering the transformation $X_2'=X_2-\Sigma_{21}\Sigma_{11}^{-1}X_1$ but things are getting too messy and I don't know how to proceed.

Please prove some hint(s) only and not a complete solution.

$\endgroup$
  • $\begingroup$ A hint about how to think of the problem: $X$ is the result of a nonsingular linear transformation of $p$ independent standard normal random variables $(Y_1,Y_2,\ldots, Y_p)$. $X^T\Sigma^{-1}X$ is essentially giving you $\sum_{i=1}^p Y_i^2$ while $X_1^T\Sigma_{11}^{-1}X_1$ is giving you $\sum_{i=1}^{p_1} Y_i^2$. The difference is thus $\sum_{i=p_1+1}^{p} Y_i^2$ which is a $\chi^2$ random variable with $p_2= p - p_1$ degrees of freedom $\endgroup$ – Dilip Sarwate Jun 14 '15 at 16:02
  • $\begingroup$ Yes, I could gather this information from the problem statement itself. But this does not help in solving the problem, does it? Or am I missing anything? $\endgroup$ – Landon Carter Jun 14 '15 at 17:34
  • $\begingroup$ Good! Mark Stone's hint should set you on the right track. $\endgroup$ – Dilip Sarwate Jun 14 '15 at 18:04
  • $\begingroup$ @DilipSarwate May I ask you any book/pdf for learning the Multivariate Normal distribution and the various properties, theorems, etc. on it? Also, it would be great if it is available on the internet for free. I read Rao but I don't like the presentation. $\endgroup$ – Landon Carter Jun 14 '15 at 18:14
  • 2
    $\begingroup$ C.R. Rao "Linear Statistical Inference And Its Applications" is a classic. It is tedious in places, but there's a lot packed in there. $\endgroup$ – Mark L. Stone Jun 14 '15 at 18:27
2
$\begingroup$

Hint (really, really good hint):

The Schur Complement is your friend.

https://en.wikipedia.org/wiki/Schur_complement

Section A.5.5, section C.4 and numerous other occurrences in http://stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf .

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Hi! So I did as follows: Suppose the Schur Decomposition is $S=\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}$. Using the formula for $\Sigma^{-1}$ which I found in Wiki, I finally have $Q=X_2^T\Sigma_{22}^{-1}X_2-(2X_1-\Sigma_{12}\Sigma_{22}^{-1}X_2)^TS^{-1}\Sigma_{12}\Sigma_{22}^{-1}X_2$. Now I know that $X_2^T\Sigma_{22}^{-1}X_2$ follows $\chi^2(p_2)$, but what can be done with the other term? It is quite weird. Will it help to consider $X_1'=X_1-\Sigma_{12}\Sigma_{22}^{-1}X_2$ so that $X_1'$ is independent of $X_2$? $\endgroup$ – Landon Carter Jun 14 '15 at 13:48
  • $\begingroup$ Actually I am not quite sure how bilinear forms like $X_1AX_2$ are distributed where $X_1$ and $X_2$ are two different vectors, maybe not of same length even. $\endgroup$ – Landon Carter Jun 14 '15 at 14:31
  • $\begingroup$ Last and final hint (otherwise I would be completely doing your homework for you, and I'm old-fashioned, because I always did all of my homework (late 70s/early 80s) by myself with no help from anyone, including T.A.s). The Schur Complement can be used to determine the conditional distribution X | X1 . $\endgroup$ – Mark L. Stone Jun 14 '15 at 15:48
  • $\begingroup$ Actually this is not a Homework problem, it came in a Semester exam. This topic will be taught to us next year, so you can say I am a beginner. Thank you for your answer. Although the final comment seemed a lot alien to me. I was thinking this can be solved using simple elementary techniques like idempotence etc. but am wrong, it seems. $\endgroup$ – Landon Carter Jun 14 '15 at 17:30
  • $\begingroup$ Schur Complement is "elementary". Anyhow, you need not know anything of the Schur Complement to solve it - there's more than one way to skin a cat, so to speak.. You could just do it yourself. But using Schur Complement, someone has already done the hard work for you, so it's easy as pie. $\endgroup$ – Mark L. Stone Jun 14 '15 at 18:29
1
$\begingroup$

There is a simple solution that was perhaps not considered.

We have

$$Q=X^T\Sigma^{-1}X-X_1^T\Sigma_{11}^{-1}X_1=X^TAX\,,$$

where $A=\Sigma^{-1}-\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}$ is a symmetric matrix.

$Q$ has a $\chi^2$ distribution if and only if $A\Sigma$ is idempotent (the 'if part' is discussed here).

Now, $$A\Sigma=I_p-\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}\Sigma$$

But $$\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}\Sigma=\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}\begin{pmatrix}\Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22}\end{pmatrix}=\begin{pmatrix}I_{p_1} & \Sigma_{11}^{-1}\Sigma_{12} \\ 0 & 0\end{pmatrix}\,,$$ which is symmetric idempotent. Hence $A\Sigma$ is also symmetric idempotent.

Degrees of freedom of $Q$ would be $\operatorname{rank}(A\Sigma)=\operatorname{tr}(A\Sigma)$. There would also be a noncentrality parameter if $\mu\ne 0$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ $A\Sigma$ need not be symmetric, so ignore that part. $\endgroup$ – StubbornAtom Jul 11 at 6:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.