2
$\begingroup$

Suppose $X$ is a $p$-dimensional vector following $N_p(\mu,\Sigma)$ distribution, where $\mu$ is $p$-dimensional and $\Sigma$ is $p\times p$. Let $X=\left(\begin{array}{ccc}X_1\\X_2\end{array} \right)$ and $\Sigma=\left( \begin{array}{ccc} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21}&\Sigma_{22} \end{array} \right)$ where $X_1$ is $p_1\times1$ and $X_2$ is $p_2\times1$ with $p_1+p_2=p$. Also we have by usual notations $\Sigma_{12}=\text{Cov}(X_1,X_2)=\Sigma_{21}^T$. Show that $$Q:=X^T\Sigma^{-1}X-X_1^T\Sigma_{11}^{-1}X_1$$follows $\chi^2(p_2)$.

I tried to proceed making the changes $Y=\Sigma^{-1/2}X$ and $Z=\Sigma_{11}^{-1/2}X_1$ but am not really sure if it is in the right direction. Also, I tried to proceed by considering the transformation $X_2'=X_2-\Sigma_{21}\Sigma_{11}^{-1}X_1$ but things are getting too messy and I don't know how to proceed.

Please prove some hint(s) only and not a complete solution.

$\endgroup$
6
  • $\begingroup$ A hint about how to think of the problem: $X$ is the result of a nonsingular linear transformation of $p$ independent standard normal random variables $(Y_1,Y_2,\ldots, Y_p)$. $X^T\Sigma^{-1}X$ is essentially giving you $\sum_{i=1}^p Y_i^2$ while $X_1^T\Sigma_{11}^{-1}X_1$ is giving you $\sum_{i=1}^{p_1} Y_i^2$. The difference is thus $\sum_{i=p_1+1}^{p} Y_i^2$ which is a $\chi^2$ random variable with $p_2= p - p_1$ degrees of freedom $\endgroup$ Jun 14, 2015 at 16:02
  • $\begingroup$ Yes, I could gather this information from the problem statement itself. But this does not help in solving the problem, does it? Or am I missing anything? $\endgroup$ Jun 14, 2015 at 17:34
  • $\begingroup$ Good! Mark Stone's hint should set you on the right track. $\endgroup$ Jun 14, 2015 at 18:04
  • $\begingroup$ @DilipSarwate May I ask you any book/pdf for learning the Multivariate Normal distribution and the various properties, theorems, etc. on it? Also, it would be great if it is available on the internet for free. I read Rao but I don't like the presentation. $\endgroup$ Jun 14, 2015 at 18:14
  • 2
    $\begingroup$ C.R. Rao "Linear Statistical Inference And Its Applications" is a classic. It is tedious in places, but there's a lot packed in there. $\endgroup$ Jun 14, 2015 at 18:27

3 Answers 3

3
$\begingroup$

Hint (really, really good hint):

The Schur Complement is your friend.

Section A.5.5, section C.4 and numerous other occurrences in "Convex Optimization", Stephen Boyd and Lieven Vandenberghe

$\endgroup$
5
  • $\begingroup$ Hi! So I did as follows: Suppose the Schur Decomposition is $S=\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}$. Using the formula for $\Sigma^{-1}$ which I found in Wiki, I finally have $Q=X_2^T\Sigma_{22}^{-1}X_2-(2X_1-\Sigma_{12}\Sigma_{22}^{-1}X_2)^TS^{-1}\Sigma_{12}\Sigma_{22}^{-1}X_2$. Now I know that $X_2^T\Sigma_{22}^{-1}X_2$ follows $\chi^2(p_2)$, but what can be done with the other term? It is quite weird. Will it help to consider $X_1'=X_1-\Sigma_{12}\Sigma_{22}^{-1}X_2$ so that $X_1'$ is independent of $X_2$? $\endgroup$ Jun 14, 2015 at 13:48
  • $\begingroup$ Actually I am not quite sure how bilinear forms like $X_1AX_2$ are distributed where $X_1$ and $X_2$ are two different vectors, maybe not of same length even. $\endgroup$ Jun 14, 2015 at 14:31
  • $\begingroup$ Last and final hint (otherwise I would be completely doing your homework for you, and I'm old-fashioned, because I always did all of my homework (late 70s/early 80s) by myself with no help from anyone, including T.A.s). The Schur Complement can be used to determine the conditional distribution X | X1 . $\endgroup$ Jun 14, 2015 at 15:48
  • $\begingroup$ Actually this is not a Homework problem, it came in a Semester exam. This topic will be taught to us next year, so you can say I am a beginner. Thank you for your answer. Although the final comment seemed a lot alien to me. I was thinking this can be solved using simple elementary techniques like idempotence etc. but am wrong, it seems. $\endgroup$ Jun 14, 2015 at 17:30
  • $\begingroup$ Schur Complement is "elementary". Anyhow, you need not know anything of the Schur Complement to solve it - there's more than one way to skin a cat, so to speak.. You could just do it yourself. But using Schur Complement, someone has already done the hard work for you, so it's easy as pie. $\endgroup$ Jun 14, 2015 at 18:29
1
$\begingroup$

There is a simple solution that was perhaps not considered.

We have

$$Q=X^T\Sigma^{-1}X-X_1^T\Sigma_{11}^{-1}X_1=X^TAX\,,$$

where $A=\Sigma^{-1}-\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}$ is a symmetric matrix.

$Q$ has a $\chi^2$ distribution if and only if $A\Sigma$ is idempotent (the 'if part' is discussed here).

Now, $$A\Sigma=I_p-\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}\Sigma$$

But $$\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}\Sigma=\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}\begin{pmatrix}\Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22}\end{pmatrix}=\begin{pmatrix}I_{p_1} & \Sigma_{11}^{-1}\Sigma_{12} \\ 0 & 0\end{pmatrix}\,,$$ which is symmetric idempotent. Hence $A\Sigma$ is also symmetric idempotent.

Degrees of freedom of $Q$ would be $\operatorname{rank}(A\Sigma)=\operatorname{tr}(A\Sigma)$. There would also be a noncentrality parameter if $\mu\ne 0$.

$\endgroup$
1
  • $\begingroup$ $A\Sigma$ need not be symmetric, so ignore that part. $\endgroup$ Jul 11, 2020 at 6:12
1
$\begingroup$

Too long for a comment, I just wanted to point out the conclusion stated in the OP's question is inaccurate, if not incorrect. The correct conclusion should be stated as follows (source: Aspects of Multivariate Statistical Theory by Muirhead, Robb J.)

enter image description here

Specifically, the conclusion cannot ignore the role of the mean vector $\mu$. OP's conclusion is correct only if $\mu = 0$. But it looks to me OP did not want to restrict himself to this special case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.