Why is Kernel Density Estimation still nonparametric with parametrized kernel?

Question

I am new to kernel density estimation (KDE), but I want to learn about it to help me calculate probabilities of outcomes in sequencing data. I watched this https://www.youtube.com/watch?v=QSNN0no4dSI as my first introduction to the subject.

As the lecturer was going over different kernels, I realized it confused me that KDE is considered non-parametric even when the kernel was being locally parameterized by points within a bandwidth.

Are the standard deviation and arithmetic mean not parameters of KDE when the kernel is the normal distribution?

Answer 1

The idea of a non-parametric density estimator is that it should be able to approximate any distribution arbitrarily closely for a large enough sample size. The class of distributions defined by the KDE method is parametric; it arguably has $n+1$ parameters for any $n$ data points (the parameters being the data points themselves and the bandwidth parameter), but if one does not accept that the means are parameters then it still has one parameter (the bandwidth). Under an appropriate estimation method for the bandwidth, as $n \rightarrow \infty$ the KDE will converge towards the true distribution of the observed sequence of values for underlying IID data.

Answer 2

Doing a little more reading, I have seen different definitions that change my perspective of non-parametric models. I believed that a non-parametric model/distribution must entirely lack parameters, but some report that non-parametric statistics may or may not have parameters. What distinguishes non-parametric statistics from parametric statistics is that they do not have a fixed or a priori distribution or model structure.

To answer the question, KDE with a normal kernel does not assume that the resulting distribution will have a particular shape, and does not assume a model structure (Ex. the number of parameters). Thus, KDE with a normal kernel is non-parametric.

Answer 3

Maybe this picture helps. Actually, the "parametric" kernels only give weights and do not strongly influence (as @conjectures says) the shape of the density estimate, in the sense that different kernels often produce very similar density estimates.

The example shows 20 purple realizations of some mixture of normals. The grey bell curves indicate the weights that each observation yields for the points at which we want density estimates ($[-4,4]$ in the picture). A kernel density estimate at some point $x$ then simply consists of stacking the weights at that point on top of each other. In the picture, we see that at $x=-2.2$, essentially only two observations contribute weights to the estimate at that point (technically, with a normal kernel, all weights are nonzero, but as the normal tails decay quickly, the weights quickly become negligible). These weights are the magenta and green bars.

That the orange and red curve agree confirms that this handmade approach to constructing a KDE just reproduces what Rs density command does.

Answer 4

KDE is about fitting a density function to data. As the result could be an arbitrary curve (from some wide family) it could be considered nonparametric.

I'm not familiar with the video you link but assume the kernel is being used to fit some local part of the overall density function's shape. Therefore the parameters of the kernel function influence but do not dictate the shape of the overall density function which is mainly determined by the data. Contrast this to a regular Normal density function where the parameters absolutely determine the shape of the density.