Estimating a sparse inverse covariance matrix with known sparsity

Question

The inverse of the covariance matrix for a distribution can be a good value for the mass matrix of a Hamiltonian monte carlo distribution.

If the distribution in question is the posterior of a Bayesian graphical model, many or most of the variables will be conditionally independent of each other. Thus, the inverse of the covariance matrix will have zeros for those pairs of variables. You can figure out the sparsity pattern of the matrix by examining the model graph.

I would like to efficiently estimate the inverse of the covariance matrix for a distribution given some IID samples from the distribution and assuming that we know its sparsity pattern.

It appears this problem is well studied for the more general case where we're trying to infer the sparsity pattern. For example, this paper is well cited and draws upon a number of other papers. However, I don't know of papers that deal with the case where the sparsity pattern is known. Any pointers?

Answer 1

Estimation of the covariance matrix with given restrictions on the inverse covariance matrix is of course a well studied problem. Restricting some entries to be 0 is an example of a linear restriction in the cone of positive semidefinite matrices.

If the distribution is multivariate normal, the inverse covariance matrix is the canonical parameter in the framework of exponential families and methods for estimation of parameters under linear hypotheses on the canonical parameter for exponential families can be used for this specific example. For decomposable graphs the estimation equations are recursively solvable and otherwise an iterative algorithm will work.

The Graphical Models book by Steffen Lauritzen (in particular, Chapter 5) provides all the details, but perhaps also too many details if the interest is only in the estimation of the (inverse) covariance matrix. The paper Gaussian Markov Distributions over Finite Graphs by Speed and Kiiviri is more directly targeting the question by the OP, and I believe it is very readable.