4
$\begingroup$

Let $$w=X^TV^{-1}X$$ Where $X$ is a vector of Normally distributed variables with covariance matrix $V$. Then $w$ is chi-squared.

I have read (without an explanation) that w has degrees of freedom equal to the rank of $V$, rather than its dimension. This seems intuitive, but is there an easy proof to show this?

Ps. I think that $V^{-1}$ should be the Moore-Penrose generalized inverse, since if $V$ has not full rank, it cannot be inverted. Is that correct?

$\endgroup$
4
$\begingroup$

Let $X \sim \mathcal N(0, V)$ with $V \in \mathbb R^{p\times p}$.

First assume $V$ is invertible so $V^{-1/2}X \sim \mathcal N(0, I)$ and therefore $$ (V^{-1/2}X)^T(V^{-1/2}X) = X^TV^{-1}X \sim \chi^2_p. $$ $V$ being invertible means it is full rank so in this case $\text{rank } V = p$ and the two coincide.


Now suppose $V$ is not invertible, i.e. it is just PSD. Let $V = Q\Lambda Q^T$ be the eigendecomposition of $V$ and let $r =\text{rank } V < p$ so there are $p-r > 0$ eigenvalues that are exactly $0$.

Consider $$ X^TV^+X = X^TQ\Lambda^+Q^TX $$ where $V^+ = Q\Lambda^+ Q^T$ is the pseudoinverse of $V$ and $(\Lambda^+)_{ii}$ is $1/\lambda_i$ if $\lambda_i \neq 0$ and $0$ otherwise.

Partition $\Lambda$ as $$ \Lambda = \left(\begin{array}{c|c} L & 0\\ \hline0& 0\end{array}\right) $$ so $L \in \mathbb R^{r\times r}$ is diagonal with all non-zero entries, and the lower right block is $(p-r)\times (p-r)$. Then $$ \Lambda^+ = \left(\begin{array}{c|c} L^{-1} & 0\\ \hline0& 0\end{array}\right). $$ Let $Y = \sqrt{\Lambda^+}Q^TX$, where $\lambda_i \geq 0$ makes $\sqrt{\Lambda^+}$ well-defined, and note that $Y^TY = X^TV^+X$. Then $$ \text{Var}(Y) = \sqrt{\Lambda^+}Q^TQ\Lambda Q^T\sqrt{\Lambda^+} $$ $$ = \left(\begin{array}{c|c} I & 0\\ \hline0& 0\end{array}\right) := H. $$ Thus $Y_{r+1} = \dots=Y_p = 0$ so $$ X^TV^+X = Y^TY = \sum_{i=1}^r Y_i^2 \sim \chi^2_r. $$


Here's the proof that this $V^+$ really is the Moore-Penrose inverse. For a matrix $A$, the Moore-Penrose inverse $A^+$ needs to satisfy the following properties:

  1. $AA^+A=A$
  2. $A^+AA^+=A^+$
  3. $AA^+$ and $A^+A$ are symmetric

For (1) and (2), since $\Lambda^+\Lambda = \Lambda\Lambda^+ = H$, and $H$ acts as the identity on $\Lambda$ and $\Lambda^+$, we have $$ VV^+V = Q\Lambda \Lambda^+\Lambda Q^T = Q\Lambda Q^T = V $$ and $$ V^+VV^+ = Q\Lambda^+ \Lambda\Lambda^+ Q^T = V^+. $$

For (3), $$ V^+V = Q\Lambda^+\Lambda Q^T = QHQ^T = Q\Lambda \Lambda^+Q^T = VV^+ $$ so $VV^+ = V^+V$ and both are symmetric.

This proves that $V^+$ is indeed the Moore-Penrose inverse of $V$.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.