How to choose a kernel for KDE There are a lot of kernels available for a univariate KDE. R uses normal by default, but the efficacy discussion seems to support the use of Epanechnikov. What should influence kernel choice for univariate exploratory analysis?
 A: This is not really a data visualization question. The information is fairly readily available online, eg http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/AV0405/MISHRA/kde.html
mentions using AMISE to select bandwidth, same approach for kernels could be used. But for EDA, you would want to work like the recommendation for histograms, re-plot with different binwidths to learn different things in the data. Sometimes using a different kernel may be helpful. The normal kernel is generally useful, and I think the bandwidth is more important than the actual kernel.
I would suggest adding tags: distributions, nonparametric. Possibly get better answers under these topics.
A: The framework of regularization theory (see Regularization Theory and Neural Networks Architectures by Girosi et. al) allows to tackle the problem of looking for a good kernel in a systematic way.
The idea is that the kernel is determined by a smoothness stabilizer which is analogous to controlling the complexity in the MDL sense, or the bias-variance error decomposition.
The idea is that you attempt to solve the problem,
$$
H(f) = \sum_{i}\left(f(x_{i})-y_{i}\right)^{2} + \lambda ||Df||^{2}
$$
where $D$ is a differential operator like for example $\frac{d^{2}}{dx^{2}}$. Now it can be proved that this results in the following solution,
$$
f(x) = \sum_{i}c_{i}G(x-x_{i})
$$
where $G$ is the Green function associated with the regularizer.
By means of cross-validation you can search for good values of $\lambda$ and the order of the differential operator.
