11
$\begingroup$

I often need make $(x,y)$ scatter plots that have many ($>10^5$) points. I've experimented with different ways of representing this many points that capture the distribution while getting around the messiness of actually putting a million points in an image. I've done the obvious things like thinning points, and plotting the local density of points as either contours or a colour map. I've found that thinning tends to be ineffective as the distributions I deal with are often exponentially peaked, so I just get the same problem in a narrower area, and local density representations are sensitive to the details of smoothing/sampling used to get the density field.

Lately I've been thinking about trying a different strategy, but I haven't been able to come up with an algorithm to make it work (which is where this question comes in). This sort of plot, which I've heard cosmologists colloquially call a 'banana plot', is the inspiration, and I think is fairly common in Bayesian analysis:

likelihood contours

Here the contours show the $1\sigma$ and $2\sigma$ confidence intervals for a combination of two parameters. In this application a 2D probability density function is generated, so making the plot is easy. I want to do something similar.

I'd like to draw contours enclosing 99%, 95% and 68% (and more generally any percentage) of the data points in an $(x,y)$ dataset. Obviously such a contour is not unique, so an extra constraint is needed. I think something like minimising the area or maximising the average density of points inside the contour should give the desired effect. Does anyone know an algorithm that would produce such a contour? I eventually want to make a plot using matplotlib, so bonus points if you happen to know such an algorithm that is implemented in python, but I'll do the implementation myself if I have to.

Not terribly familiar with the tags here on CV.SE, so if I'm missing any obvious ones please suggest/edit them in.

Improve this question
$\endgroup$
9
  • $\begingroup$ You may be interested in bag plots. $\endgroup$
    – Andy W
    Commented Aug 22, 2013 at 18:13
  • 2
    $\begingroup$ Also see highest density region (if it is ok that the contour is not entirely enclosed in one polygon). $\endgroup$
    – Andy W
    Commented Aug 22, 2013 at 18:44
  • 2
    $\begingroup$ A closely related question is asked and answered at stats.stackexchange.com/questions/63447/…. In this case, if you are not firmly wedded to the idea of imposing an extra constraint explicitly, an answer is easy to come by: just compute a kernel density estimate on a fine grid and select the largest 68%, 95%, or 99% from the cumulative sum of its sorted values: they form a region within the grid. A smoothed version of the boundary of that region will do. $\endgroup$
    – whuber ♦
    Commented Aug 22, 2013 at 19:56
  • $\begingroup$ @whuber I've played around with density methods (similar to the one you linked) and they work alright, but I wanted to try something that is independent of any smoothing kernel choice or similar. $\endgroup$
    – Kyle
    Commented Aug 22, 2013 at 20:49
  • 1
    $\begingroup$ I hear you. My concern is that this way of framing the question is likely to lead to huge computational difficulties. Controlling a density bandwidth (or something in that spirit) is easy to do, computationally reasonable, and should be sufficiently flexible to meet most needs. $\endgroup$
    – whuber ♦
    Commented Aug 22, 2013 at 21:18

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.