I was wondering whether anyone could point me to some references that discuss the interpretation of the elements of the inverse covariance matrix, also known as the concentration matrix or the precision matrix.

I have access to Cox and Wermuth's Multivariate Dependencies, but what I'm looking for is an interpretation of each element in the inverse matrix. Wikipedia states: "The elements of the precision matrix have an interpretation in terms of partial correlations and partial variances," which leads me to this page. Is there an interpretation without using linear regression? IE, in terms of covariances or geometry?

  • 4
    $\begingroup$ did you read the entire Wikipedia page? There is a section on geometry and on conditional independence for the normal distribution. You can find more in this book. $\endgroup$
    – NRH
    May 14, 2011 at 7:18
  • $\begingroup$ @NRH The geometry is explained in the partial correlation page, which I'm not even sure how it relates to the concentration matrix yet. Does that graphical models book have an explanation of the elements of the concentration matrix? Thanks! $\endgroup$ May 16, 2011 at 19:09
  • $\begingroup$ see answer below. $\endgroup$
    – NRH
    May 16, 2011 at 20:26
  • 2

5 Answers 5


There are basically two things to be said. The first is that if you look at the density for the multivariate normal distribution (with mean 0 here) it is proportional to $$\exp\left(-\frac{1}{2}x^T P x\right)$$ where $P = \Sigma^{-1}$ is the inverse of the covariance matrix, also called the precision. This matrix is positive definite and defines via $$(x,y) \mapsto x^T P y$$ an inner product on $\mathbb{R}^p$. The resulting geometry, which gives specific meaning to the concept of orthogonality and defines a norm related to the normal distribution, is important, and to understand, for instance, the geometric content of LDA you need to view things in the light of the geometry given by $P$.

The other thing to be said is that the partial correlations can be read of directly from $P$, see here. The same Wikipedia page gives that the partial correlations, and thus the entries of $P$, have a geometrical interpretation in terms of cosine to an angle. What is, perhaps, more important in the context of partial correlations is that the partial correlation between $X_i$ and $X_j$ is 0 if and only if entry $i,j$ in $P$ is zero. For the normal distribution the variables $X_i$ and $X_j$ are then conditionally independent given all other variables. This is what Steffens book, that I referred to in the comment above, is all about. Conditional independence and Graphical models. It has a fairly complete treatment of the normal distribution, but it may not be that easy to follow.

  • 1
    $\begingroup$ Sorry I'm a bit confused wrt to the Wikipedia formula for partial correlation; I've seen several implementations taking ${\bf\color{red} -} \frac{p_{ij}}{ \sqrt{p_{ii} p_{jj}}}$ (with a minus sign). Are you sure that the Wikipedia formula is correct? $\endgroup$
    – Jonathan H
    Jul 1, 2015 at 17:07
  • 2
    $\begingroup$ @Sh3ljohn, you are perfectly right. There is a minus missing in the Wikipedia formula. $\endgroup$
    – NRH
    Jul 2, 2015 at 12:20
  • $\begingroup$ Isn't the first answer really talking more about the Fisher information than the precision matrix? I mean they coincide in the really special/nice Gaussian case, but they don't coincide generally. Obviously the two concepts are related (Cramer-Rao lower bound, asymptotic distribution of MLE, etc.) but it doesn't seem helpful to conflate them (specifically I came to this question looking for his question about how to distinguish Fisher information and the inverse correlation matrix). $\endgroup$ Dec 1, 2017 at 4:32

I like this probabilistic graphical model to illustrate NRH's point that the partial correlation is zero if and only if X is conditionally independent from Y given Z, with the assumption that all involved variables are multivariate Gaussian (the property does not hold in the general case):

enter image description here

(the $y_i$ are Gaussian random variables; ignore T and k)

Source: David MacKay's talk on Gaussian Process Basics, 25th minute.


The interpretation based on partial correlations is probably the most statistically useful, since it applies to all multivariate distributions. In the special case of the multivariate Normal distribution, zero partial correlation corresponds to conditional independence.

You can derive this interpretation by using the Schur complement to get a formula for the entries of the concentration matrix in terms of the entries of the covariance matrix. See http://en.wikipedia.org/wiki/Schur_complement#Applications_to_probability_theory_and_statistics


Covariance matrix can represent relations between all variables while inverse covariance shows the relations of elements with their neighbors (as wikipedia said partial/pair wise relations).

I borrow the following example from here in 24:10. Imagine 5 masses are connected together and vowelling around with 6 springs. The covariance matrix would contain correlation of all masses, if one goes right, others can also goes right, but the inverse covariance matrix shows the relation of those masses that are connected by same springs (neighbors) and it contains many zeros and it is not necessary positive.

  • 1
    $\begingroup$ Where is this explained in the video? It's an hour long. Thanks! $\endgroup$ May 16, 2011 at 19:07
  • $\begingroup$ you are right, its on 24:10, I think thats the best example to understand the nature of cov matrix and its inverse $\endgroup$
    – Areza
    May 21, 2011 at 11:44

Bar-Shalom and Fortmann (1988) make mention of the inverse covariance in the context of Kalman filtering as follows:

...[T]here is a recursion for the inverse covariance (or information matrix)

$\mathbf{P}^{-1}(k+1|k+1) = \mathbf{P}^{-1}(k+1|k) + \mathbf{H}'(k+1) \mathbf{R}^{-1}(k+1)\mathbf{H}(k+1)$

...Indeed, a complete set of prediction and update equations, known as the information filter[8, 29, 142], can be developed for the inverse covariance and a transformed state vector $\mathbf{P}^{-1}\hat{\mathbf{x}}$.

The book is indexed at Google.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.