When thinking about a histogram as an estimate of the density function, is it reasonable to think of the bin size as a parameter that constrains the local structure of that function?
Also, is there a better way to articulate this reasoning?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ I think you could look at kernel density estimation $\endgroup$– seanv507Apr 8, 2018 at 11:16
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
Yes, this is a reasonable way to think about it (assuming the histogram is normalized to obtain a proper pdf). Bin width constrains the smoothness of the density estimate (speaking loosely, since histograms are discontinuous functions). It controls the extent to which finer structure can be modeled, and also the extent to which random fluctuations in the data affect the estimate. It plays a similar role as the kernel width in kernel density estimation, and hyperparameters that control leaf size in decision trees.
To be a little more specific, bin width is a hyperparameter that controls the bias variance tradeoff. Reducing bin width decreases bias because it allows a finer representation--histograms with narrower bins form a richer class of functions that can better approximate the true/underlying distribution. But, it increases variance because fewer data points are available for estimating the height of each bin--histograms with narrower bins are more sensitive to random fluctuations in the data, and will vary more over datasets drawn from the same underlying distribution. A good bin width balances these opposing effects to give a density estimate that better matches the underlying distribution.
For more detail see:
Scott (1979). On optimal and data-based histograms.
Shalizi (2009). Estimating Distributions and Densities [course notes]
answered Apr 9, 2018 at 12:11
$\begingroup$
$\endgroup$
Kernel density estimators are oftentimes rationalised as a "continuous" version of a histogram. Many books on nonparametric kernel estimation also discuss histograms. See, e.g., chapter 2 in Racine, Jeffrey S. "Nonparametric econometrics: A primer." Foundations and Trends® in Econometrics 3.1 (2008): 1-88.
$\begingroup$
$\endgroup$
It is reasonable, because what you're doing by putting samples in bins is approximating the data. In my experience depending on your goal and data available, those bins can vary drastically and have a big impact on how the data is handled further. For some cases you might not need a lot bins or maybe you lack data, so you can still see the general curve. On the other side if the approximation is too strong you can miss out on some details, like local mins and maxs or the structure. For example you can take the following function:
And compare the hist for 100 and 8 bins
There's a clear difference between the structure complexity. If we're talking about the density function, of course you should choose the second option for a more smooth curve without such extreme values as on the first image
Usually I prefer to use Freedman–Diaconis rule as a rule of thumb to choose the default number of bins and then tune it considering the task.
