# Why is Kernel Density Estimation still nonparametric with parametrized kernel?

I am new to kernel density estimation (KDE), but I want to learn about it to help me calculate probabilities of outcomes in sequencing data. I watched this https://www.youtube.com/watch?v=QSNN0no4dSI as my first introduction to the subject.

As the lecturer was going over different kernels, I realized it confused me that KDE is considered non-parametric even when the kernel was being locally parameterized by points within a bandwidth.

Are the standard deviation and arithmetic mean not parameters of KDE when the kernel is the normal distribution?

• For the kernel itself the mean is $0$; but we do specify the bandwidth, $h$, which determines the amount of smoothing; it doesn't mean that we have an underlying parametric model for the density $f$ (even though conditionally on the sample, it's modelled as a finite mixture of Gaussians, the number of parameters in the model is not fixed, but changes with sample size) Dec 9, 2015 at 21:55

Doing a little more reading, I have seen different definitions that change my perspective of non-parametric models. I believed that a non-parametric model/distribution must entirely lack parameters, but some report that non-parametric statistics may or may not have parameters. What distinguishes non-parametric statistics from parametric statistics is that they do not have a fixed or a priori distribution or model structure.

To answer the question, KDE with a normal kernel does not assume that the resulting distribution will have a particular shape, and does not assume a model structure (Ex. the number of parameters). Thus, KDE with a normal kernel is non-parametric.

• This is correct - more or less. In KDE you do not assume any parametric (functional) form of the data. You can contrast that with trying to estimate the density if you assume normality - here you directly make a parametric/functional asummption, and the entire shape of the curve depends on it. There will be a lot of difference between one where you assume normality vs. (say) a Chi-square. However, it makes very little difference if you chose a Gaussian kernel or a Epanechnikov kernel for KDE. Both include an important parameter, namely the bandwidth but its still nonparametric fitting. Dec 9, 2015 at 19:58

Maybe this picture helps. Actually, the "parametric" kernels only give weights and do not strongly influence (as @conjectures says) the shape of the density estimate, in the sense that different kernels often produce very similar density estimates.

The example shows 20 purple realizations of some mixture of normals. The grey bell curves indicate the weights that each observation yields for the points at which we want density estimates ($[-4,4]$ in the picture). A kernel density estimate at some point $x$ then simply consists of stacking the weights at that point on top of each other. In the picture, we see that at $x=-2.2$, essentially only two observations contribute weights to the estimate at that point (technically, with a normal kernel, all weights are nonzero, but as the normal tails decay quickly, the weights quickly become negligible). These weights are the magenta and green bars.

That the orange and red curve agree confirms that this handmade approach to constructing a KDE just reproduces what Rs density command does. KDE is about fitting a density function to data. As the result could be an arbitrary curve (from some wide family) it could be considered nonparametric.

I'm not familiar with the video you link but assume the kernel is being used to fit some local part of the overall density function's shape. Therefore the parameters of the kernel function influence but do not dictate the shape of the overall density function which is mainly determined by the data. Contrast this to a regular Normal density function where the parameters absolutely determine the shape of the density.

The idea of a non-parametric density estimator is that it should be able to approximate any distribution arbitrarily closely for a large enough sample size. The class of distributions defined by the KDE method is parametric, with $$n+1$$ parameters for any $$n$$ data points (the parameters being the data points themselves and the bandwidth parameter). As $$n \rightarrow \infty$$ the KDE will converge towards any distribution for underlying IID data.