Say I make histograms H1, H2, H4 ...of the same set of data with bins 1, 2, 4 ... wide. Then the bins containing a given $x$ have counts and averages

n1 av1 in H1,
n2 av2 in H2,
n4 av4 in H4 ...

How should one weight these to estimate data(x) ?
One possibility would be $\Sigma w_j \text{av}_j / \Sigma w_j$ with $w_j = n_j / j$, for the first two non-zero $n_j$.

The general goals are to fill holes where bins are empty, and to smooth bins with many data points.

(H2 etc. can either be built together with H1, or on-the-fly from H1 as needed: a space-time tradeoff.)

Added, trying to clarify: there are two related subquestions:

  • local vs global:
    If almost all the terms in $\Sigma K( x_j\, \text{near}\, x )$ are 0, there must be a better way. (For 1D it hardly matters, but in 2D or 3D, $N^2$, $N^3$ get big quickly.)
    One way of finding $x_j$ near $x$ (to sum $K$ at those only) is to grow shells in a regular grid -- poor for clumpy or hole-y data. Another is to find $N_{\text{near}}$ nearest neighbours in a Kd tree -- but what should $N_{\text{near}}$ be ?

  • adaptive histograms ?
    A method that adapts to clumpy data, as TINs (triangulations) adapt to terrains, would be nice.

(Years later) the magic words are "multiresolution" or "multiscale"; see e.g.
scholar allintitle: (multiscale|multiresolution) (histogram | estimation) .
Does anyone know of comparisons with conventional KDE on real data, 2d and up ?

  • 7
    $\begingroup$ What's wrong with conventional kernel density estimation? $\endgroup$ – onestop Oct 7 '10 at 12:08
  • $\begingroup$ Correct me, conventional KDE uses all the input data points for each estimate(x) -- quite unpractical for many points. I'm looking for ways to combine a) get some nearby points (a general problem), b) weight the values from this small sample $\endgroup$ – denis Oct 8 '10 at 10:54
  • 7
    $\begingroup$ If you use a kernel with a finite support (e.g., the quadratic Epanechnikov kernel), then only nearby points contribute anything to a kde. Even with a Gaussian kernel, the contribution of points more than 3 bandwidths from x is negligible. I would definitely use kde rather than your proposed approach. $\endgroup$ – Rob Hyndman Oct 9 '10 at 22:39
  • $\begingroup$ @Rob, thanks; I'll try to clarify the question, and split off "adaptive histograms" as a separate question. $\endgroup$ – denis Oct 11 '10 at 16:40

I would see each histogram as a different model (parametrized by the width). Fitting a smoothing spline or some other kind of smoother for each of the models is simple.

You can then do model selection (such as cross-validation) to choose the histogram width that gives the best results, or do model stacking to fit least-squares weights on the models.

However, why not directly smooth the data instead of clustering it into histogram bars first? There are finite-window width kernels that don't use the entire dataset for prediction at a given point. Practicality and speed depends on what you are really trying to obtain, but I am sure there exist simpler solutions.

  • 2
    $\begingroup$ Agree with last para. The Gaussian kernel uses all the data points but other commonly-used kernels (Epanechnikov, biweight, cosine, Parzen, triangular, ...) are finite-width, i.e. they are defined to be zero outside a finite interval. R's density() function defaults to Gaussian but Stata's -kdensity- command defaults to Epanechnikov. $\endgroup$ – onestop Oct 9 '10 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.