Complex regression plot in R

I need to draw a complex graphics for visual data analysis. I have 2 variables and a big number of cases (>1000). For example (number is 100 if to make dispersion less "normal"):

x <- rnorm(100,mean=95,sd=50)
y <- rnorm(100,mean=35,sd=20)
d <- data.frame(x=x,y=y)


1) I need to plot raw data with point size, corresponding the relative frequency of coincidences, so plot(x,y) is not an option - I need point sizes. What should be done to achieve this?

2) On the same plot I need to plot 95% confidence interval ellipse and line representing change of correlation (do not know how to name it correctly) - something like this:

library(corrgram)
corrgram(d, order=TRUE, lower.panel=panel.ellipse, upper.panel=panel.pts)


but with both graphs at one plot.

3) Finally, I need to draw a resulting linar regression model on top of this all:

r<-lm(y~x, data=d)
abline(r,col=2,lwd=2)


but with error range... something like on QQ-plot:

but for fitting errors, if it is possible.

So the question is:

How to achieve all of this at one graph?

-

Does the picture below look like what you want to achieve?

do.it <- function(df, type="confidence", ...) {
require(ellipse)
lm0 <- lm(y ~ x, data=df)
xc <- with(df, xyTable(x, y))
df.new <- data.frame(x=seq(min(df$x), max(df$x), 0.1))
pred.ulb <- predict(lm0, df.new, interval=type)
pred.lo <- predict(loess(y ~ x, data=df), df.new)
plot(xc$x, xc$y, cex=xc$number*2/3, xlab="x", ylab="y", ...) abline(lm0, col="red") lines(df.new$x, pred.lo, col="green", lwd=1.5)
lines(df.new$x, pred.ulb[,"lwr"], lty=2, col="red") lines(df.new$x, pred.ulb[,"upr"], lty=2, col="red")
lines(ellipse(cor(df$x, df$y), scale=c(sd(df$x),sd(df$y)),
centre=c(mean(df$x),mean(df$y))), lwd=1.5, col="green")
invisible(lm0)
}

set.seed(101)
n <- 1000
x <- rnorm(n, mean=2)
y <- 1.5 + 0.4*x + rnorm(n)
df <- data.frame(x=x, y=y)

# take a bootstrap sample
df <- df[sample(nrow(df), nrow(df), rep=TRUE),]

do.it(df, pch=19, col=rgb(0,0,.7,.5))


And here is the ggplotized version

produced with the following piece of code:

xc <- with(df, xyTable(x, y))
df2 <- cbind.data.frame(x=xc$x, y=xc$y, n=xc$number) df.ell <- as.data.frame(with(df, ellipse(cor(x, y), scale=c(sd(x),sd(y)), centre=c(mean(x),mean(y))))) library(ggplot2) ggplot(data=df2, aes(x=x, y=y)) + geom_point(aes(size=n), alpha=.6) + stat_smooth(data=df, method="loess", se=FALSE, color="green") + stat_smooth(data=df, method="lm") + geom_path(data=df.ell, colour="green", size=1.2)  It could be customized a little bit more by adding model fit indices, like Cook's distance, with a color shading effect. - @chl +1, nice graph, and short code. – mpiktas Mar 5 '11 at 10:44 @mpiktas Thanks. This led me to realize I didn't work with the right sample, in fact :-) – chl Mar 5 '11 at 10:50 Almost looks like the one I need, but with real numbers I faced the following problems: 1) df.new <- data.frame(x = seq(min(x), max(x), 0.1)) is better. 2) Ellipse is drawn at 0;0 position, which is not correct and its size is also strange (too small). Also tryed library(car) dataEllipse(df$x, df$y, levels=0.95:1, lty=2) but it drops all. 3) The curve (like on correlogramm) is missing. I almost reproduced it by calling library(car) cr.plots(m0) but the range of data is incorrect. Use first 2 lines from my code instead of yours to reproduce. – Yuriy Petrovskiy Mar 5 '11 at 14:36 @Yuriy Ok, I will update my code (no need to make any edit in the meantime), but I cannot see how we could get overlap with real-valued random variates with your$(x,y)\$ setting; this is the reason why I use boostrap with replacement (this ensures that ~ 2/3 of the original units are present). car::dataEllipse does provide the same facilities than in the ellipse package, but it is probably less easy to customize. I guess the superimposed curve is just a loess, so it is not difficult to add. –  chl Mar 5 '11 at 14:56
@Tal The interpretation of the ellipse is the same as in the corrgram package: it shows 95% pairwise confidence region assuming a bivariate normal distribution centered on the mean and scaled by SD(x) and SD(y). I'm not a big fan of this when used in a scatterplot, though. But see Murdoch & Chow, A graphical display of large correlation matrices, Am Stat (1996) 50:178, or Friendly, Corrgrams: Exploratory displays for correlation matrices, Am Stat (2002) 56:316. –  chl Mar 6 '11 at 16:01

For point 1 just use the cex parameter on plot to set the point size.

For instance

x = rnorm(100)
plot(x, pch=20, cex=abs(x))


To have multiple graphs in one plot use par(mfrow=c(numrows, numcols)) to have an evenly spaced layout or layout to make more complex ones.

-
+1 for the tip about cex, but I think the OP wants all stuff on the same plotting region, not on separate ones. –  chl Mar 5 '11 at 10:28
Ahh... now I understand the question. Well, then he can just use curve or points` to overplot the three graphs ;) –  nico Mar 5 '11 at 10:54