Appropriate Kernel for Kernel density estimation

Question

Given $\mathcal G$ an arbitrary family of density distributions, and a large number of samples from an unknown distribution $P$, is there any method to find the most appropriate kernel in $\mathcal G$ and then use it to estimate $P$ ?

Answer 1

Depending on what is meant by 'most appropriate', the short answer likely is that there is no method that is mutually agreed upon to lead to a most appropriate kernel. Also, the choice of the smoothing parameter is usually more important than the kernel that is chosen, and the rest of my answer will be highlighting this point.

I think my answer will make more sense if I first explain the idea behind kernel density estimation. Let $\tilde{x} = (x_1, x_2, \dots, x_n)$ be your sample of size $n$ from $P$. Assuming $P$ is univariate and continuous, the quickest way to get an idea of the shape of the distribution of $\tilde{x}$ is to plot a histogram of $\tilde{x}$ and compute some summary statistics (measures of central tendency and dispersion). Histograms separate the data into 'bins' which is usually the number of bars you see in your histogram, and each bin has a width.

Kernel density estimation is a generalization of histogram density estimation. If you think about constructing a histogram with bin width $h$ from your sample $\tilde{x}$, then a density estimate for $x_i \in \tilde{x}$ is

$$\hat{f} (x_i) = \frac{k}{2 h n},$$

where $k$ is the number of sample points in $(x_i - h, x_i + h)$. The estimator $\hat{f} (x_i)$ can be rewritten as

$$\hat{f} (x_i) = \frac{1}{2hn} \sum_{j=1}^n I \bigg( \bigg|\frac{x_i - x_j}{h} \bigg| < 1 \bigg).$$

Now define the weight function $w(t) = \frac{1}{2}I(|t| < 1)$, with properties $\int_{-1}^1 w(t)dt = 1$ and $w(t) \geq 0$. Then $w(t)$ is called the rectangular kernel (hence kernel density is a generalization). Kernel density estimation replaces the weight function $w(t)$ in $\hat{f} (x_i)$ with a kernel function, $K(\cdot )$, so that

$$\hat{f} (x_i) = \frac{1}{nh} \sum_{j=1}^n K \bigg( \frac{x_i - x_j}{h} \bigg)$$.

Now, here's the important takeaway. Think back to constructing the histrogram, histograms with a smaller bin width $h$ will have noisier histograms than a histogram with a larger bin width. Notice that the kernel density estimate still has the parameter $h$. In context of kernel density, $h$ is usually called the 'smoothing parameter' or 'bandwidth' by those in applied fields.

Because your kernel density estimate uses $h$ to control the smoothness of the density estimate, the choice of $h$ is usually more important than the choice of kernel. If $h$ is too small, then your estimated distribution may have multiple false modes. On the other hand, if $h$ is too large, then you may smooth away important features of your distribution.

A suggested method for finding the 'best' value of $h$ is to find $h$ that minimizes the mean integrated squared error (or MISE), which is the expected value of the difference between the observed density and the density estimate. Different methods exist for finding the optimal $h$ for different kernels and can be found in the literature.

If you're still wondering which kernel is most appropriate at this point (and have time to try multiple kernels), one thing you can try is compare the MISE for different kernels each with their own optimal $h$, and use the MISE values and graphs of each optimal density estimate to make your decision regarding the most appropriate kernel.

References: Rizzo, Maria L. "Statistical computing with R", 2007. Givens, Geof H. and Hoeting, Jennifer A. "Computational Statistics", 2012.