I am new to kernel density estimation (KDE), but I want to learn about it to help me calculate probabilities of outcomes in sequencing data. I watched this https://www.youtube.com/watch?v=QSNN0no4dSI as my first introduction to the subject.

As the lecturer was going over different kernels, I realized it confused me that KDE is considered non-parametric even when the kernel was being locally parameterized by points within a bandwidth.

Are the standard deviation and arithmetic mean not parameters of KDE when the kernel is the normal distribution?

  • 1
    $\begingroup$ For the kernel itself the mean is $0$; but we do specify the bandwidth, $h$, which determines the amount of smoothing; it doesn't mean that we have an underlying parametric model for the density $f$ (even though conditionally on the sample, it's modelled as a finite mixture of Gaussians, the number of parameters in the model is not fixed, but changes with sample size) $\endgroup$ – Glen_b Dec 9 '15 at 21:55

Doing a little more reading, I have seen different definitions that change my perspective of non-parametric models. I believed that a non-parametric model/distribution must entirely lack parameters, but some report that non-parametric statistics may or may not have parameters. What distinguishes non-parametric statistics from parametric statistics is that they do not have a fixed or a priori distribution or model structure.

To answer the question, KDE with a normal kernel does not assume that the resulting distribution will have a particular shape, and does not assume a model structure (Ex. the number of parameters). Thus, KDE with a normal kernel is non-parametric.

  • $\begingroup$ This is correct - more or less. In KDE you do not assume any parametric (functional) form of the data. You can contrast that with trying to estimate the density if you assume normality - here you directly make a parametric/functional asummption, and the entire shape of the curve depends on it. There will be a lot of difference between one where you assume normality vs. (say) a Chi-square. However, it makes very little difference if you chose a Gaussian kernel or a Epanechnikov kernel for KDE. Both include an important parameter, namely the bandwidth but its still nonparametric fitting. $\endgroup$ – Repmat Dec 9 '15 at 19:58

Maybe this picture helps. Actually, the "parametric" kernels only give weights and do not strongly influence (as @conjectures says) the shape of the density estimate, in the sense that different kernels often produce very similar density estimates.

The example shows 20 purple realizations of some mixture of normals. The grey bell curves indicate the weights that each observation yields for the points at which we want density estimates ($[-4,4]$ in the picture). A kernel density estimate at some point $x$ then simply consists of stacking the weights at that point on top of each other. In the picture, we see that at $x=-2.2$, essentially only two observations contribute weights to the estimate at that point (technically, with a normal kernel, all weights are nonzero, but as the normal tails decay quickly, the weights quickly become negligible). These weights are the magenta and green bars.

That the orange and red curve agree confirms that this handmade approach to constructing a KDE just reproduces what Rs density command does.

enter image description here


KDE is about fitting a density function to data. As the result could be an arbitrary curve (from some wide family) it could be considered nonparametric.

I'm not familiar with the video you link but assume the kernel is being used to fit some local part of the overall density function's shape. Therefore the parameters of the kernel function influence but do not dictate the shape of the overall density function which is mainly determined by the data. Contrast this to a regular Normal density function where the parameters absolutely determine the shape of the density.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.