5
$\begingroup$

I am trying hard to do the following and have already spent a few hours in vain:

I wanted to do the scatter plot. But given the high dispersion on those dots, I would like to bin the x-axis and then for each bin of the x-axis, plot the quantiles of the y-values of the data points in each bin:

  1. Uniform bin size on the x-axis;
  2. Equal number of observations in each bin;

(These two are separate cases.)

How to do that in R? I guess for the sake of prettyness, I'd better do it in ggplot2?

The origin of this problem was that a plain scatter plot with too many points with high dispersion generated too many points flying all over places.
We are trying to smooth the charts a bit...
Any good recommendations?

How about "plot the quantiles of each bin"?

But how are the quantiles plotted? Shall I specify 50% quantile, etc?

[p.s. Update 3/11/2011]: I am trying the following following R-help posts:

DAT <- data.frame(x = runif(1000, 0, 20), y = rnorm(1000))
DAT$xbin <- with(DAT, cut(x, seq(0, 20, 2)))

p <- ggplot(DAT, aes(x = x, y = y)) + geom_point(alpha = 0.2) +
stat_quantile(aes(colour = ..quantile..), quantiles = seq(0.05, 0.95,
by=0.05)) + facet_wrap(~ xbin, scales = "free")
print(p)

My questions are:

1) How do I make it "equal number of points" in each bin along the x-axis? i.e. the original number 2 requirement in my question?

2) And also, no matter how I changed the quantiles = seq(0.05, 0.95, by=0.05)) line, the number of lines in each bin and the number of legends on the right side of the each plot are different...

What's the catch? Am I missing something here?

I thought the number of quantile lines and the number of legends should be exactly the same, no?

Improve this question
$\endgroup$
6

3 Answers 3

Reset to default
7
$\begingroup$

You can to do this in the new version of ggplot2 (0.9).

You can try it out:

library(ggplot2) #make sure the newest is installed

df <- data.frame(v1 = runif(1000), v2 = runif(1000))

bin.plot<-qplot(data=df,
                x=v1,
                y=v2,
                z=v2)

Basic plot

bin.plot+stat_summary_hex(fun=function(z)length(z))

Plot with hexagonal binning

bin.plot+stat_summary2d()(fun=function(z)length(z))

Plot with rectangular binning

These may also be of interest if you want to bin only on one variable

geom_violin
geom_dotplot

You can also start by binning your data and then jitter it.

The release notes of ggplot2 0.9: http://cloud.github.com/downloads/hadley/ggplot2/guide-col.pdf

For development versions of ggplot2

#library(devtools)
#dev_mode()
#install_github("ggplot2")
#library(ggplot2)
Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I think ggplot2 0.9 already came up on CRAN :-) $\endgroup$
    – chl
    Commented Mar 11, 2012 at 19:53
  • $\begingroup$ thanks but it doesn't work on my R... $\endgroup$
    – Luna
    Commented Mar 12, 2012 at 0:23
  • 2
    $\begingroup$ @Luna What is not working, precisely?! $\endgroup$
    – chl
    Commented Mar 12, 2012 at 7:26
6
$\begingroup$

You may want to look at these two entries from 'SAS and R':

http://sas-and-r.blogspot.com/2011/07/example-91-scatterplots-with-binning.html
http://sas-and-r.blogspot.com/2011/07/example-92-transparency-and-bivariate.html

They cover the use of binning, transparency and bivariate kernel density estimators for scatter plots of large amounts of data. They might serve as decent starting points.

I'm rather biased against ggplot2, so I won't comment on whether or not you need to use it for prettyness - I find the figures in these entries to be perfectly appealing.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Why are you biased against ggplot2? $\endgroup$
    – mark999
    Commented Mar 10, 2012 at 7:52
  • 4
    $\begingroup$ @mark999 I...just don't like most plots made in ggplot2. Essentially I think the message people have come out with is "ggplot means pretty graphs" when it should have been "thinking actively about how you visualize things means pretty graphs". A busy, unclear plot in ggplot2 isn't any more useful. $\endgroup$
    – Fomite
    Commented Mar 12, 2012 at 0:41
  • $\begingroup$ @EpiGrad Sounds more like you don't like the skills/aesthetic judgements of most people you see using ggplot2, than ggplot2 itself. $\endgroup$
    – joran
    Commented Mar 13, 2012 at 1:50
  • $\begingroup$ @joran A bit of both. I also don't particularly care for ggplot's aesthetics, but I recognize that that's hopelessly subjective. $\endgroup$
    – Fomite
    Commented Mar 13, 2012 at 19:49
  • 1
    $\begingroup$ Part (if not all) of the reason ggplot2 is so popular is because of the implementation of the grammar of graphics in an easily readable/understandable fashion. I agree you can make crappy charts using any program, and I come across default settings I don't like all the time (in alot of different stat software/programs). Whether the program gives you the flexibility to change them to your liking is a key point then (and ggplot2 certainly does this). $\endgroup$
    – Andy W
    Commented Mar 15, 2012 at 12:47
3
$\begingroup$

It's not really an answer to your question about binning one easy solution in ggplot2 to deal with large amount of data in scatterplots is to use the alpha parameter to set some transparency

> df <- data.frame(v1 = rnorm(100000), v2 = rnorm(100000))
> ggplot(df, aes(x=v1, y=v2)) + geom_point(alpha = .01) + theme_bw()

Result

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ Note that the possibility of using hexbin with ggplot2 is discussed in this response: stats.stackexchange.com/a/14972/930. I, for one, believe that transparency is helpful in case there's some overlapping in the data and jittering doesn't help, but for large dataset this is not the solution. $\endgroup$
    – chl
    Commented Mar 10, 2012 at 11:31
  • 1
    $\begingroup$ @chl I'd assert that it depends on what you want to show for a large data set. For example, transparency is very good at showing areas of high density, while still preserving outlying values, which things like binning and KDE sometimes struggle with. $\endgroup$
    – Fomite
    Commented Mar 12, 2012 at 0:42
  • $\begingroup$ I don't understand the usefulness of "hexbin"... it's very different from what we need in this problem... thanks anyway! $\endgroup$
    – Luna
    Commented Mar 12, 2012 at 2:02
  • $\begingroup$ @EpiGrad Agree but that really depends on sample size: over plotting is likely to obscure subtile pattern even with transparency. Other issue: vectorized output, like PDF, will be almost useless. I submitted the above code to lattice with hexbin::panel.hexbinplot: this is just a 15 Ko PDF file, as compared to the 5.5 Mo file generated when using simple points with 50% transparency. $\endgroup$
    – chl
    Commented Mar 12, 2012 at 7:11
  • 1
    $\begingroup$ Another general point (which I don't think has been made), but is still applicable to this situation. Frequently I see people not reduce the size of the points. For example ggplot(df, aes(x=v1, y=v2)) + geom_point(size = .01, alpha = .1) + theme_bw() with Gaël's data. $\endgroup$
    – Andy W
    Commented Mar 15, 2012 at 12:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.