19
$\begingroup$

I am working on creating a website, which displays the census data for a user selected Polygons & would like to graphically show the distribution of various parameters (one graph per parameter).

The data usually has the following properties:

  1. The sample size tend to be large (say around 10,000 data points)
  2. The range in values tends to be quire large (for example, the minimum population can be less than 100 & the maximum can be something like 500,000)
  3. q1 usually is close to the minimum (say 200) while q2 & q3 will be within 10,000
  4. It doesn't look anything like a normal distribution

I am not a statistician and hence my description might not be exactly clear.

I would like to show this distribution on a graph, which will be seen by citizens (the layman, if you like).

I would have best liked to use a histogram, but it is not possible due to the large range of values, due to which making bins is not really easy & straight forward.

From what little I know about statistics, a box plot is what is often used to show this kind of data, but I feel that for a layperson, deciphering the Box plot is not easy.

What are my options to show this data in an easy to understand manner?

$\endgroup$
  • $\begingroup$ what exactly are you displaying? It is not clear for me what kind of data your one data point represents. $\endgroup$ – mpiktas Dec 31 '10 at 7:49
  • 1
    $\begingroup$ How about a kernel density plot? statmethods.net/graphs/density.html $\endgroup$ – Roman Luštrik Dec 31 '10 at 9:14
  • $\begingroup$ @mpiktas: My data is Census data for villages. My website will allow the user to select an area on the map, and then will find all the villages in that area. The census data for a village consists of various values like: Male Population, Female population, Average household income etc for that village. I hope to show the data distribution for a particular value (eg: Total Population) for all the villages falling in the user selected area. $\endgroup$ – Devdatta Tengshe Dec 31 '10 at 9:25
13
$\begingroup$

A boxplot isn't that complicated. After all, you just need to compute the three quartiles, and the min and max which define the range; a subtlety arises when we want to draw the whiskers and various methods have been proposed. For instance, in a Tukey boxplot values outside 1.5 times the inter-quartile from the first or third quartile would be considered as outliers and displayed as simple points. See also Methods for Presenting Statistical Information: The Box Plot for a good overview, by Kristin Potter. The R software implements a slightly different rule but the source code is available if you want to study it (see the boxplot() and boxplot.stats() functions). However, it is not very useful when the interest is in identifying outliers from a very skewed distribution (but see, An adjusted boxplot for skewed distributions, by Hubert and Vandervieren, CSDA 2008 52(12)).

As far as online visualization is concerned, I would suggest taking a look at Protovis which is a plugin-free js toolbox for interactive web displays. The examples page has very illustrations of what can be achieved with it, in very few lines.

$\endgroup$
  • 3
    $\begingroup$ I work in biological research. I know some colleagues (I mean, people with a PhD) who cannot really grasp boxplots. I would not use them to target a general audience. $\endgroup$ – nico Dec 31 '10 at 11:17
  • 1
    $\begingroup$ @nico That's a fair point. But, this is not a reason not to use efficient graphical summary. A schematic illustration of what a boxplot actually does might help the reader. $\endgroup$ – chl Dec 31 '10 at 11:53
  • 1
    $\begingroup$ it really depends on what the target audience is and what the aim of the site is. Explaining boxplots would definitely help, but still, some people struggle a lot with the concept of distribution. $\endgroup$ – nico Dec 31 '10 at 12:00
  • $\begingroup$ @nico Yes, I agree. Although boxplot is not mentioned in A Tour through the Visualization Zoo -- but these are for large and complex data sets, I simply like it and I'm sorry to see that it is not much used in experimental sciences. Superimposing raw data is a way to help the reader visualizing the distribution. $\endgroup$ – chl Dec 31 '10 at 12:17
  • 1
    $\begingroup$ I know! I always try to "convert" my colleagues to boxplots, at least when it comes to writing papers, making presentations etc., but sometimes is though! $\endgroup$ – nico Dec 31 '10 at 12:25
11
$\begingroup$

You might also want to have a look at beanplots.

alt text

[Source]

Implemented in R package by Peter Kampstra.

$\endgroup$
7
$\begingroup$

I'd suggest you persevere with histograms. They're much more widely understood than the alternatives. Use a log scale to cope with the large range of values. Here's an example I cooked up in a couple of minutes in Stata: Histogram with log scale on the value axis
I admit that the x-axis numerical labels weren't entirely straightforward or automatic, but as you're building a website I'm sure your programming skills are up to the challenge!

$\endgroup$
  • $\begingroup$ Good point. Histograms ( or density plots with experiment with bandwidth) are a great solution here. $\endgroup$ – suncoolsu Dec 31 '10 at 12:51
  • $\begingroup$ You are completely right, that the Histogram is the most understood way to show a distribution. I will try to make histograms with both the axes in log scale. $\endgroup$ – Devdatta Tengshe Jan 3 '11 at 3:47
  • 2
    $\begingroup$ I'm only suggesting using a log scale for the x-axis. I don't think a log scale for the frequency axis would be a good idea, as then the shaded area of each bar of the histogram wouldn't be proportional to the number of observations. $\endgroup$ – onestop Jan 3 '11 at 9:06
5
$\begingroup$

Here is a matlab function for plotting multiple histograms side-by-side in 2D as an alternative to box-plot. See the picture on the top. And here is another one

The density strip is another alternative to box-plot. It is a shaded monochrome strip whose darkness at a point is proportional to the probability density of the quantity at that point. This is an R implementation of the density strip

$\endgroup$
  • 1
    $\begingroup$ (+1) Forgot about that. It might be handy. $\endgroup$ – chl Dec 31 '10 at 10:04
  • 1
    $\begingroup$ Just found an ungated PDF version of Displaying Uncertainty With Shading. $\endgroup$ – chl May 9 '11 at 12:24
  • $\begingroup$ @chl: that link does not work $\endgroup$ – kjetil b halvorsen Jan 11 at 14:46
4
$\begingroup$

How about using quantiles? It is not necessary to present a graph then, only a table. For village census I think the users will be most interested how many there are villages of certain size, so giving for example deciles will tell them them information such as $x\%$ of all the villages are smaller than the certain number. For deciles $x=0,10,20,...,100$. You can graph this table with the percents on a x-axis and the deciles on the y-axis.

$\endgroup$
  • 3
    $\begingroup$ Quoting a friend of mine: if you want to "hide" something in a paper, put it in the text rather then in a figure. If you want to make sure nobody ever reads it put it in a table! ;) Just joking of course, but having a website with interactive maps for users to click etc. all of that to get a table... well that would be disappointing! $\endgroup$ – nico Dec 31 '10 at 11:28
  • $\begingroup$ @nico, yeah but sometimes tables are much more informative than graphs. I for example prefer table instead of a bad graph. In this case the table still can be represented by graph, and I suggested quantiles because they do not have problems with outliers. $\endgroup$ – mpiktas Dec 31 '10 at 11:46
  • $\begingroup$ That is what I am currently doing(Showing the deciles on a graph), but after showing it to some of our target audience, we received feedback, that the graphs were not easy to understand. $\endgroup$ – Devdatta Tengshe Jan 1 '11 at 5:04
2
$\begingroup$

If you are targeting the general population (i.e. a non statistical-savvy audience) you should focus on eye-candy rather than statistical accuracy.

Forget about boxplots, let alone violin plots (I personally find them very difficult to read)! If you'd ask the average street man what a quantile is, you would mostly get some wide eyed silence...

You should use barplots, bubble charts, maybe some pie charts (brrrr). Forget about error bars (although I would put SD in text somewhere where applicable).

Use colors, shapes, thick lines, 3D. You should make each chart unique and immediately easy to understand, even without having to read all the legends/axes etc. Make a smart use of maps by coloring them.

Information is beautiful is a very good resource to get ideas. Look at this chart for instance: Caffeine and Calories: anyone can understand it, and it's pleasing to the eye.

And, of course, have a look at Edward Tufte's work.

$\endgroup$
  • $\begingroup$ Note I wasn't suggesting he use violin plots for his applications, but a histogram with logarithmically spaced bins. Violin plots was the answer to the question in the title (which was rather different to the question in the post itself). $\endgroup$ – Dikran Marsupial Dec 31 '10 at 11:21
  • 3
    $\begingroup$ You will probably like Many Eyes, dataviz, datavisualization.ch, and Ideas2evidence, to name a few. $\endgroup$ – chl Dec 31 '10 at 12:12
2
$\begingroup$

I rather like violin plots myself, as this gives an idea of the shape of the distribution. However if the large range of values is the issue, then maybe it would be best to plot the log of the data rather than the raw values, that would then make choosing the box sizes for histograms etc. As the display is for laymen, don't mention logs and mark the axis 10, 100, 1000, 10000, 100000, 1000000 etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.