Nowadays the Dirichlet process mixture (DPM) seems to be the default Bayesian approach for density estimation. My question is why not simply use the kernel density estimate (KDE) to model the density? One can surely use a Bayesian sampler (e.g. MCMC) to infer the bandwidth of KDE and thus make it "Bayesian". I would argue that posterior draws from Bayesian KDE are easier to check convergence than DPM. What would be the theoretical benefits of DPM over KDE?
After some research and thinking, here would be my own tentative answer to the question I posted; just in case someone else is interested in this question.
Given n data points, KDE uses a mixture of n kernels to approximate the "true" density while DPM, in finite samples, typically ends up with a smaller mixture even though it theoretically uses an infinite number of mixtures. Moreover, KDE fixes the mixing weights at 1/n while DPM allows it to follow the stick breaking process. Assuming Gaussian kernels, it is easy to see that KDE fixes the means of the kernels at the data points while DPM estimates them from data. Taken together, DPM could argue that it is more parsimonious (using a smaller mixture) and is more flexible (not fixing mixture weights and kernel parameters) than KDE. The empirical findings in http://mlg.eng.cam.ac.uk/pub/pdf/GoeRas10.pdf contains a comparison of the performance of KDE and DPM and seems to suggest that DPM performs better.
(To Tim's comments: Rob Hyndman has a paper on estimating KDE bandwidth https://robjhyndman.com/publications/bandwidth-selection-for-multivariate-kernel-density-estimation-using-mcmc/. There are probably more papers on that topic. As for convergence, depending on the algorithm for DPM, the number of kernels as well as their labels change across MCMC iterations and hence makes it challenging to check the draws' convergence.)