1
$\begingroup$

First off, This is not an R question - it is a conceptual question, so there is no need to perform any code.

Problem: I'm trying to invert a large dimensional covariance matrix of p features. For large p, even using a graphical LASSO with a regularization parameter, inverting this matrix takes a long time. In the past, I have elected instead to use the Meinhausen-Buhlmann approximation to speed the inversion up, using R's glasso() method from the glasso package. You can specify the approximation by setting the parameter approx to TRUE: glasso(cov.matrix, rho=0.15, approx=TRUE). This makes the inversion of even a large covariance matrix, very speedy.

However, I'm looking at my resulting graphical structure and I worry I don't trust the approximation, because the clustering of the nodes isn't very organized/interpretable. I expect to see clusters based on annotation groups each node belongs to.

Do I have reason to distrust the Meinhausen-Bulhmann approximation? What assumptions does it make? The original paper is a bit out of my depth, so I would appreciate help in digesting how the approximation works from experts/smarter people.

Thanks for any insights!

:-)

$\endgroup$
1
$\begingroup$

Estimation quality of a precision matrix in high dimensions $p>>n$ through any procedure is dependent on assumptions--in short there's no free lunch, and I rather doubt there is a useful estimator that is minimax in the space of all precision matrices.

In the case of the graphical lasso, the assumption is sparsity. If your precision matrix is not sparse (say it was low-rank, instead), then the graphical lasso or neighborhood selection is unlikely to produce anything sensible.

Now, supposing your precision is sparse, and your data are indeed Gaussian: the theory for Meinshausen-Buhlmann (also known as neighborhood selection) merely relies on observing that the conditional distribution of each variable, conditioned on all others implies a linear regression relationship. This follows from the properties of the conditional distribution of a multivariate normal distribution.

So the conditional, neighborhood selection procedure and the graphical lasso are estimating the same parameter. I will try to dig up the reference, but in practice they are pretty similar in efficiency in finite samples as well. So I would suggest that if you aren't happy with the networks you are seeing from neighborhood selection, the Gaussian model is probably mis-specified in some sense, rather than statistical inefficiency. What do the marginal and bi-variate distributions look like? Are they plausibly Gaussian?

$\endgroup$
1
  • $\begingroup$ You definitely know what you're talking about - thanks so much for all the name drops! I am new to graduate level stats. Isn't a graphical lasso purposed to estimate the precision matrix (or inverse covariance matrix)? If so, how can you know what the rank of the real precision matrix is (besides waiting FOREVER for its computation to finish)? I set the regularization parameter rho to 0.35, and this creates a sparse precision matrix (99% of elements in precision matrix are 0). However, I plotted my data in a qqplot and did Shapiro, and it's definitely not Gaussian. What are my options? $\endgroup$ Feb 11 '16 at 19:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.