How to present the gain in explained variance thanks to the correlation of Y and X?

Question

I'm searching how to (visually) explain simple linear correlation to first year students.

The classical way to visualize would be to give an Y~X scatter plot with a straight regression line.

Recently, I came by the idea of extending this type of graphics by adding to the plot 3 more images, leaving me with: the scatter plot of y~1, then of y~x, resid(y~x)~x and lastly of residuals(y~x)~1 (centered to the mean)

Here is an example of such a visualization:

And the R code to produce it:

set.seed(345)
x <- runif(50) * 10
y <- x +rnorm(50)


layout(matrix(c(1,2,2,2,2,3 ,3,3,3,4), 1,10))
plot(y~rep(1, length(y)), axes = F, xlab = "", ylim = range(y))
points(1,mean(y), col = 2, pch = 19, cex = 2)
plot(y~x, ylab = "", )
abline(lm(y~x), col = 2, lwd = 2)

plot(c(residuals(lm(y~x)) + mean(y))~x, ylab = "", ylim = range(y))
abline(h =mean(y), col = 2, lwd = 2)

plot(c(residuals(lm(y~x)) + mean(y))~rep(1, length(y)), axes = F, xlab = "", ylab = "", ylim = range(y))
points(1,mean(y), col = 2, pch = 19, cex = 2)

Which leads me to my question: I would appreciate any suggestions on how this graph can be enhanced (either with text, marks, or any other type of relevant visualizations). Adding relevant R code will also be nice.

One direction is to add some information of the R^2 (either by text, or by somehow adding lines presenting the magnitude of the variance before and after the introduction of x) Another option is to highlight one point and showing how it is "better explained" thanks to the regression line. Any input will be appreciated.

Cheers, Tal

Answer 1

Here are some suggestions (about your plot, not about how I would illustrate correlation/regression analysis):

• The two univariate plots you show in the right and left margins may be simplified with a call to rug();
• I find more informative to show a density plot of $X$ and $Y$ or a boxplot, at risk of being evocative of the idea of a bi-normality assumption which makes no sense in this context;
• In addition to the regression line, it is worth showing a non-parametric estimate of the trend, like a loess (this is good practice and highly informative about possible local non linearities);
• Points might be highlighted (with varying color or size) according to Leverage effect or Cook distances, i.e. any of those measures that show how influential individual values are on the estimated regression line. I'll second @DWin's comment and I think it is better to highlight how individual points "degrade" goodness-of-fit or induce some kind of departure from the linearity assumption.

Of note, this graph assumes X and Y are non-paired data, otherwise I would stick to a Bland-Altman plot ($(X-Y)$ against $(X+Y)/2$), in addition to scatterplot.

Answer 2

Not answering to your exact question, but the followings could be interesting by visualizing one possible pitfall of linear correlations based on an answer from stackoveflow:

par(mfrow=c(2,1))

set.seed(1)
x <- rnorm(1000)
y <- rnorm(1000)
plot(y~x, ylab = "", main=paste('1000 random values (r=', round(cor(x,y), 4), ')',  sep=''))
abline(lm(y~x), col = 2, lwd = 2)

x <- c(x, 500)
y <- c(y, 500)
cor(x,y)
plot(y~x, ylab = "", main=paste('1000 random values and (500, 500) (r=', round(cor(x,y), 4), ')',  sep=''))
abline(lm(y~x), col = 2, lwd = 2)

@Gavin Simpson's and @bill_080's answer also includes nice plots of correlation in the same topic.

Answer 3

I'd have two two-panel plots, both have the xy plot on the left, and a histogram on the right. In the first plot, a horizontal line is placed at the mean of y and lines extend from this to each point, representing the residuals of y values from the mean. The histogram with this simply plots these residuals. Then in the next pair, the xy plot contains a line representing the linear fit and again vertical lines representing the residuals, which are represented in a histogram to the right. Keep x axis of the histograms constant to highlight the shift to lower values in the linear fit relative to the mean "fit".

Answer 4

I think what you propose is good, but I would do it in three different examples

1) X and Y are completely unrelated. Simply remove "x" from the r code that generates y (y1<-rnorm(50))

2) The example you posted (y2 <- x+rnorm(50))

3) The X are Y are the same variable. Simply remove "rnorm(50)" from the r code that generates y (y3<-x)

This would more explicitly show how increasing the correlation decreases the variability in the residuals. You would just need to make sure that the vertical axis doesn't change with each plot, which may happen if you're using default scaling.

So you could compare three plots r1 vs x, r2 vs x and r3 vs x. I am using "r" to indicate the residuals from the fit using y1, y2, and y3 respectively.

My R skills in plotting are quite hopeless, so I can't offer much help here.