2
$\begingroup$

I was wondering, is there a general rule or a "golden rule" that sets the appropriate bin size as a function of statistical parameters such as sample size, mean, median, mode, standard deviation, etc. when the data is not known to follow a certain distribution?

The reason why I ask is because I have a relatively large data set (about 1.5 million values). However, the range of these values is from 0 to about 0.6. Nothing is known about the distribution of this data (i.e, is it "normal") because the data itself is derived from an experimental process of which its mechanism is not categorized fully as of yet. I am worried that certain bin sizes will encompass values that would distort the shape of my histogram. I understand that eventually you have to create a hardline boundary for your bins as too fine of a bin control can give you a very awkward shape, a shape that may explain too much diversity even though there might not be. However, if the lines are too ambiguous, you could have a very general shape and you could lose information on data stratification.

Here are some statistical parameters for my data set:

Range = ~0.6

Min = 0

Max = ~0.6

St. Dev = 0.063

n = $1.5*10^6$

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Wikipedia list some different approaches but with over a million data points you could just choose bin widths of $0.01$ starting at $0$, so about $60$ bins and see if that causes an issue with the histogram $\endgroup$
    – Henry
    Commented Jul 15, 2015 at 19:42
  • 1
    $\begingroup$ One universal rule should be to draw any histogram in more than one way. See stats.stackexchange.com/a/51753. $\endgroup$
    – whuber ♦
    Commented Jul 15, 2015 at 20:31
  • $\begingroup$ There's a plethora of rules ... and to my eye they tend to oversmooth, some vastly so. What properties did you want this "golden rule" to have? $\endgroup$
    – Glen_b
    Commented Jul 16, 2015 at 1:22
  • $\begingroup$ I want it to be able to not jam meaningful data (kind of ambiguous, I know) into other bins while still giving me enough stratification that I can see differences. $\endgroup$
    – Alvin Nunez
    Commented Jul 16, 2015 at 20:05

2 Answers 2

Reset to default
1
$\begingroup$

With 1.5 million observations, the choice of bin size should be irrelevant. In fact, one could use density smoothing estimates to have something like a continuous histogram to represent their data. Regardless, the number of total overall bins should simply be a function of how finely you wish to present these data. 10 bins, visually, can be a lot to take in but can present complicated distributions that are either skewed or multimodal. 6 bins is good for presenting a global mode and ranges.

Improve this answer
$\endgroup$
3
  • 4
    $\begingroup$ The bin size could be highly relevant if the distribution is multimodal. Thus, the density plot recommendation is good, but the suggestions that 10 or even six bins would work seem to have no foundation. $\endgroup$
    – whuber ♦
    Commented Jul 15, 2015 at 20:06
  • $\begingroup$ @whuber it depends on the size of the graphic. There's such a thing as simply too many information pieces. Most histograms only really need the "1 column width" as in about 4 inches on an 8.5" X 11" piece of US standard letter paper. $\endgroup$
    – AdamO
    Commented Jul 15, 2015 at 20:07
  • $\begingroup$ My experience with histograms suggests other considerations should take precedence over the physical size, such as bin populations, underlying distribution, purpose of showing the histogram, and the intended audience (roughly in order of priority). Even a tiny postage-stamp histogram at (say) 300 dpi is capable of accurately showing 200 bins. As Tufte has emphasized, it's not the amount of information that matters, but how it is displayed--and, within limits, a great deal of information can effectively be conveyed in a single graphic. $\endgroup$
    – whuber ♦
    Commented Jul 15, 2015 at 20:29
1
$\begingroup$

it is called, Sturges' rule and is to set the number of intervals as close as possible to $1 + \log_2(N)$

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ You write as if there were only one such rule of thumb, but in fact there are a large number of them--and they can produce considerably different recommendations for bin sizes. At a minimum, then, you ought to explain why you think this is the rule to follow, what assumptions it makes, and what it is intended to achieve in terms of histogram quality. $\endgroup$
    – whuber ♦
    Commented Jul 15, 2015 at 20:04
  • $\begingroup$ For the best of my knowledge, it is the simplest way of choosing bins and is used in many software. although there are alternative ways but I guess this can answer the question well. Thanks @whuber $\endgroup$
    – TPArrow
    Commented Jul 15, 2015 at 22:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.