6
$\begingroup$

I was wondering if anyone knows of any really productive ways to graph/map/plot/visualize covariance matrix data. I am currently using levelplot() in R. It's very basic, and does the job, but we're looking for something better. We're also using R Shiny so that there is more interaction between the user and the data, but that also seems pretty basic. Please keep in mind my matrix is 2600x2600, so very large!

If you have any neat, creative ideas, or know of packages that I would be interested in.

$\endgroup$
  • 1
    $\begingroup$ See ?heatmap. For instance, heatmap(cor(x)). $\endgroup$ – AdamO Jun 6 '13 at 21:34
  • $\begingroup$ Thanks @AdamO! I actually have used heatmap already. I like levelplot better, but they're both good (just basic). Thanks, though! $\endgroup$ – Darla Jun 7 '13 at 13:02
6
$\begingroup$

Look at the corrplot package for R. It has several options for visualizing correlation matrices.

$\endgroup$

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

  • $\begingroup$ Thanks for your quick responses, guys! I looked into the corrplot package. Unfortunately I'm kind of having the same issue as I did with the corrgram package. My data is too large to really get someone out of those graphs. We really want something interactive. Maybe when you have this intense correlation graph, you can use your mouse to select where you want to like, zoom in on. Know of anything like that? $\endgroup$ – Darla Jun 7 '13 at 13:18
3
$\begingroup$

I'm not sure about any R packages that do this, but my favourite approach (and I feel a much more informative approach) to visualization of covariance matrices are using correlation networks. Basically, I trim the covariance matrix using the Glasso algorithm (I'll explain why later), then use a force-directed algorithm to produce a network for the correlations. The size of the nodes are the variances. Ex:

correlation network

larger image

Remember, this is a network, so the axis mean nothing; it's all about neighbourhoods and distances. The red lines are high correlations, and the blue lines are negative correlations. We can spot clusters of variables, and second and third order correlations better than a matrix approach.

I trim the covariance matrix, by using a penalty term (see Glasso algorithm) for two reasons: reduce estimation variance, and reduce number of lines which improves visualization.

FYI, the above was generated using this Python script.

$\endgroup$
  • $\begingroup$ Please reproduce the Python script ;-), thanks! $\endgroup$ – Néstor Jun 7 '13 at 16:35
  • $\begingroup$ that is pretty neat. I love the approach! Have you ever done this on much larger data? My matrix is 2600x2600, so it might be very busy. can you do things like zoom in on it and such? Also, can this be done in R? I'll have to research that. $\endgroup$ – Darla Jun 7 '13 at 16:55
  • $\begingroup$ Never for anything quite that large. (The most I think was ~60-80). It would work, but I can't guarantee the execution would be quick. For that many variables, I would have a very high penalty term in the Glasso algo, so most correlations are 0. $\endgroup$ – Cam.Davidson.Pilon Jun 7 '13 at 17:02
  • $\begingroup$ I'm not sure about R. I would switch to Python. The script I linked can be used with a csv actually, by calling python gl_tools.py csv_name.csv out.png -a <alpha> but you need some libraries in python installed. $\endgroup$ – Cam.Davidson.Pilon Jun 7 '13 at 17:03
  • $\begingroup$ btw, you have 2600 variables, but how many data instances do you have? $\endgroup$ – Cam.Davidson.Pilon Jun 7 '13 at 17:07
1
$\begingroup$

If you're using shiny and have the ability to make interactive tools, then just develop the covariance plot as a ggplot object so that you can add hover, dblclick, and brush tools for zooming and showing pop-up information.

Here's an intro to creating different correlation like plots in R with ggplot2: GGPlot2 CorrMatrix. And here's an intro to adding a pop-up with hover tools using ggplot objects in Shiny: Shiny Plot Hover Tool.

Alternatively, you may find the Plotly has some nice features for graphing the info you want, which also has easy-to-incorporate, interactive tools. For example: Plotly Heatmaps.

If you have questions on the ggplot method, feel free to message me. I've actually just recently developed a similar app with hover and zoom tools.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.