10
$\begingroup$

I am analyzing a dataset with many parameters (say, 50-200) and I am interested in looking at relations between variables (e.g. in terms of 2-variable scatter plots or 2d histograms). However, for this number of parameters it seems unfeasible to draw a 200x200 array of plots (unless I print it and hang on a wall).

On the other hand, doing only a correlation matrix does not give all information about 2-variable relations.

Is there a way (library or workflow) to explore 2-variable relations for many variables?

I am especially interested in showing results to others (perhaps after some data preprocessing). E.g. something interactive in JavaScript, were I can see scatter-plot matrix for selected fields from a correlation matrix.

By scatter-plot matrix I mean something like that:

enter image description here

(taken from pandasplotting blog; aviable in Python / Pandas, R, D3.js, etc).

Improve this question
$\endgroup$
4
  • 4
    $\begingroup$ You didn't put it clear what you're after. Do you want to see the cloud, every data point? Do you want to see all the bivariate facets at once? $\endgroup$
    – ttnphns
    Jul 19, 2013 at 6:55
  • $\begingroup$ @ttnphns I want to see all data points OR them in some aggregated form (e.g. 2d histograms). I don't need to see all things are once (as it makes sense for <15 variables, but not 200). And yes, I am aware that the question is a bit open-ended. A close-ended version is "Is there a JS library to show scatter plots & histograms when mouse hovers on the respective pixel at a correlation matrix? Or should I write one? :)" I wrote a more general one, as perhaps there are some better workflows for dealing with the problem. $\endgroup$
    – Piotr Migdal
    Jul 19, 2013 at 15:36
  • $\begingroup$ It is possible to organize the matrix of scatterplots in the form of olap cube, so that you see only one or several plots at a time and can switch between them. Unfortunately, I don't know a specific program or code to do graphical olap cubes. $\endgroup$
    – ttnphns
    Jul 19, 2013 at 15:47
  • 1
    $\begingroup$ Possibly related: How to extract information from a scatterplot matrix when you have large N, discrete data, & many variables? $\endgroup$
    – gung - Reinstate Monica
    Apr 11, 2015 at 5:47

2 Answers 2

Reset to default
7
$\begingroup$

Explore relations between variables is quite vague, but two of the more general goals of examining scatterplots like this I would guess are;

  • Identify underlying latent groups (of variables or cases).
  • Identify outliers (in univariate, bivariate or multivariate space).

Both reduce the data into more manageble summaries, but have different goals. Identify latent groups one typically reduces the dimensions in the data (e.g. via PCA) and then explores whether variables or cases cluster together in this reduced space. See for example Friendly (2002) or Cook et al. (1995).

Identifying outliers can either mean fitting a model and plotting the deviations from the model (e.g. plotting residuals from a regression model) or reducing the data into its main components and only highlight points that deviate from either the model or the main body of data. E.g. boxplots in one or two dimensions typically only show individual points that are outside of the hinges (Wickham & Stryjewski, 2013). Plotting residuals has the nice property that it should flatten plots (Tukey, 1977), so any evidence of relationships in the remaining point cloud is "interesting". This question on CV has some excellent suggestions of identifying multivariate outliers.

A common way to explore such large SPLOMS is to not plot all of the individual points, but some type of simplified summary and then maybe points that deviate largely from this summary, e.g. confidence ellipses, scagnostic summaries (Wilkinson & Wills, 2008), bivariate box-plots, contour plots. Below is an example of plotting ellipses that define the covariance and superimposing a loess smoother to describe the linear association.

Corrgram Ellipses
(source: statmethods.net)

Either way, a real sucessful, interactive plot with so many variables would likely need intelligent sorting (Wilkinson, 2005) and a simple way to filter out variables (in addition to brushing/linking capabilities). Also any realistic dataset would need to have the capabilities to transform axis (e.g. plot the data on logarithmic scale, transform the data by taking roots etc.). Good luck, and don't stick with only one plot!

Citations

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Thanks! My goal is to explore data before I run PCA, as variables can be related in a non-linear way and require rescaling (or other processing). $\endgroup$
    – Piotr Migdal
    Jul 19, 2013 at 18:43
  • $\begingroup$ The example corrgram ellipse + loess smoother should still be pertinent then (or change the covar. ellipse with some other measure-polygon like the bivariate boxplot), and might help ID non-linear associations. A good follow up question might be if its possible to ID non-linear associations after data reduction via PCA (or something like that). $\endgroup$
    – Andy W
    Jul 19, 2013 at 19:00
1
$\begingroup$

You could consider using the shiny interface from the pairsD3 R package, which provides a way to interact with scatter plot matrices.

An example with the iris data set:

install.packages("pairsD3")
require("pairsD3")
shinypairs(iris)

Source: https://github.com/garthtarr/pairsD3

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.