I have a positive definite symmetric covariance matrix which looks like this: enter image description here

A, B, C, D and E, F, G are MATRICES, also positive definite symmetric covariance

What is the inverse of such a matrix? More Specifically I am trying to find simplifying steps for less computational complexity


The inverse of a block (or partitioned) matrix is given by

$$ \left[ \begin{array}{cc} M_{11} & M_{12} \\ M_{21} & M_{22} \end{array} \right] ^{-1} = \left[ \begin{array}{cc} K_1^{-1} & -M_{11}^{-1} M_{12}K_2^{-1} \\ -K_2^{-1} M_{21} M_{11}^{-1} & K_2^{-1} \end{array} \right], $$

where $K_1 = M_{11} - M_{12} M_{22}^{-1} M_{21}$ and $K_2 = M_{22} - M_{21} M_{11}^{-1} M_{12}$. When the matrix is block diagonal, this reduces to $$ \left[ \begin{array}{cc} M_{11} & 0 \\ 0 & M_{22} \end{array} \right] ^{-1} = \left[ \begin{array}{cc} M_{11}^{-1} & 0 \\ 0 & M_{22}^{-1} \end{array} \right]. $$

These identities are in The Matrix Cookbook. The fact that the inverse of a block diagonal matrix has a simple, diagonal form will help you a lot. I don't know of a way to exploit the fact that the matrices are symmetric and positive definite.

To invert your matrix, let $M_{11} = \left[ \begin{array}{ccc} A & 0 & 0 \\ 0 & B & 0 \\ 0 & 0 & C \end{array} \right]$, $M_{12} = M_{21}' = \left[ \begin{array}{c} E \\ F \\ G \end{array} \right]$, and $M_{22} = D$.

Recursively apply the block diagonal inverse formula gives $$ M_{11}^{-1} = \left[ \begin{array}{ccc} A & 0 & 0 \\ 0 & B & 0 \\ 0 & 0 & C \end{array} \right]^{-1} = \left[ \begin{array}{ccc} A^{-1} & 0 & 0 \\ 0 & B^{-1} & 0 \\ 0 & 0 & C^{-1} \end{array} \right]. $$

Now you can compute $C_1^{-1}$, $M_{11}^{-1}$, and $K_2^{-1}$, and plug into the first identity for the inverse of a partitioned matrix.

  • $\begingroup$ Ahh, the old Schur Complement. But you have to be careful on numerical stability - they can be real bastards. You need to think about where to do explicit inversion vs. where to do an equation solve to get the pieces you need. Of course, the real question is what use is to be made of the inverse covariance matrix, and is an explicit inverse really needed. Then you could compare operation count and numerical stability for various methods, to include "straightforward" methods not making using of the Schur complement. Nevertheless, +1 for getting the ball rolling. $\endgroup$ – Mark L. Stone May 24 '16 at 1:57
  • $\begingroup$ @MarkL.Stone is there a way to exploit teh fact that these matrices are symmetric positive definite $\endgroup$ – Wis May 24 '16 at 2:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.