Graphical LASSO Interpretation

Question

I have a conceptual question about graphical LASSO interpretation. I have been using the huge package in R to estimate an association network for a matrix of node attributes. My understanding of graphical LASSO is that you are performing a penalized regression where each vector of attributes is regressed on all other vectors of attributes. A pair of vectors $i$ and $j$ then have a tie in the resulting network if $\beta_{ij}, \beta_{ji} \neq 0 $. In most of the applications I've seen, the resulting network is interpreted in terms of similarity. This paper, for example, identifies a network of senators where senators with similar voting records are in each other's neighborhood. This is intuitive and this is what I find in my own applications.

My question is: why is this the case if a pair of vectors are included in the network when $\beta_{ji} \neq 0$? Vectors $ij$ should then be included if $\beta <0$ as well as when $\beta >0$. The network of association would include ties between highly similar and highly dissimilar vectors, both positive and negative associations. So why is it that the networks this method generates are primarily networks of similarity / positive association? Why don't we also see links between perfectly dissimilar vectors? I would greatly appreciate any help with this question.

Answer 1

Structure estimation in graphical models is a big field of statistics in it's own right. Two common methods are

• Neighbourhood Selection, Meinhausen and Buhlmann, 2006
• Graphical Lasso, originally proposed by Yuan and Lin and subject to various future practical extensions and theoretical analysis e.g. Mazumder and Hastie, Chandrasekaran et al.

What you are describing sounds like the Meinhausen-Buhlmann neighbourhood selection scheme. However, I suppose the graphical lasso function inhuge implements some version of the graphical lasso algorithm - which is related to neighbourhood selection - but different.

The references above are pretty scary - I think the introduction in Statistical Learning with Sparsity will be more friendly. This mentions a nice interpretation of the graphical lasso as doing neighbourhood selection on all nodes simultaneously.

On to why networks are primarily of positive association, I believe this is just a quirk of real-world data. One would expect similar things to behave in similar ways - this might be very common; one might expect different things to behave somewhat independently; a negative correlation suggests more complicated system dynamics that one might expect to be fairly rare (and correspondingly more interesting). Does that sound reasonable?