How to extract information from a scatterplot matrix when you have large N, discrete data, & many variables?

Question

I'm playing around with the breast cancer dataset and created a scatterplot of all attributes to get an idea for which ones have the most effect on predicting the class malignant (blue) of benign (red).

I understand that the row represents x axis and column represents y axis but I can't see what observations I can make about the data or the attributes from this scatterplot.

I'm looking for some help to interpret/make observations about the data from this scatterplot or if I should be using some other visualization to visualize this data.

R code I used

link   <- "http://www.cs.iastate.edu/~cs573x/labs/lab1/breast-cancer-wisconsin.arff"
breast <- read.arff(link)
cols   <- character(nrow(breast))
cols[] <- "black"
cols[breast$class == 2] <- "red"
cols[breast$class == 4] <- "blue"
pairs(breast, col=cols)

Answer 1

I'm not sure if this is of any help for you, but for primary EDA I really like the tabplot package. Gives you a good sense of what possible correlations there may be within your data.

install.packages("tabplot")
tableplot(breast) # gives you the unsorted image below
tableplot(breast, sortCol="class") # gives you a sorted image according to class

Answer 2

There are a number of issues that make it difficult or impossible to extract any usable information from your scatterplot matrix.

You have too many variables displayed together. When you have lots of variables in a scatterplot matrix, each plot becomes too small to be useful. The thing to notice is that many plots are duplicated, which wastes space. Also, although you do want to see every combination, you don't have to plot them all together. Notice that you can break a scatterplot matrix into smaller blocks of four or five (a number that is usefully visualizable). You just need to make multiple plots, one for each block.

Since you have a lot of data at discrete points in the space, they end up stacking on top of each other. Thus, you cannot see how many points are at each location. There are several tricks to help you deal with this.

1. The first is to jitter. Jittering means adding a small amount of noise to the values in your dataset. The noise is taken from a uniform distribution centered on your value plus or minus some small amount. There are algorithms for determining an optimal amount, but since your data come in whole units from one to ten, $.5$ seems like a good choice.
2. With so much data, even jittering will make the patters hard to discern. You can use colors that are highly saturated, but largely transparent to account for this. Where there is a lot of data stacked on top of each other, the color will become darker, and where there is little density, the color will be lighter.
3. For the transparency to work, you will need solid symbols to display your data, whereas R uses hollow circles by default.

Using these strategies, here is some example R code and the plots made:

# the alpha argument in rgb() lets you set the transparency
cols2 = c(rgb(red=255, green=0, blue=0,   alpha=50, maxColorValue=255),
          rgb(red=0,   green=0, blue=255, alpha=50, maxColorValue=255) )
cols2 = ifelse(breast$class==2, cols2[1], cols2[2])
# here we jitter the data
set.seed(6141)  # this makes the example exactly reproducible
jbreast = apply(breast[,1:9], 2, FUN=function(x){ jitter(x, amount=.5) })
jbreast = cbind(jbreast, class=breast[,10])  # the class variable is not jittered

windows()  # the 1st 5 variables, using pch=16
  pairs(jbreast[,1:5], col=cols2, pch=16)

windows()  # the 2nd 5 variables
  pairs(jbreast[,6:10], col=cols2, pch=16)

windows()  # to match up the 1st & 2nd sets requires more coding
  layout(matrix(1:25, nrow=5, byrow=T))
  par(mar=c(.5,.5,.5,.5), oma=c(2,2,2,2))
  for(i in 1:5){
    for(j in 6:10){
      plot(jbreast[,j], jbreast[,i], col=cols2, pch=16, 
           axes=F, main="", xlab="", ylab="")
      box()
      if(j==6 ){ mtext(colnames(jbreast)[i], side=2, cex=.7, line=1) }
      if(i==5 ){ mtext(colnames(jbreast)[j], side=1, cex=.7, line=1) }
      if(j==10){ axis(side=4, seq(2,10,2), cex.axis=.8) }
      if(i==1 ){ axis(side=3, seq(2,10,2), cex.axis=.8) }
    }
  }

Answer 3

It is difficult to visualize more than 3-4 dimensions in a single plot. One option would be to use principal components analysis (PCA) to compress the data and then visualize it in the main dimensions. There are several different packages in R (as well as the base prcomp function) that make this syntactically easy (see CRAN); interpreting the plots, loadings, is another story, but I think easier than a 10 variable ordinal scatterplot matrix.