1
$\begingroup$

Background: Kernel Density Estimators (KDE), given $n$ i.i.d. samples of a random variable $X$, and when chosen with the appropriate window size $h^*$, have an asymptotic mean square integrated error (AMISE) of between $n^{-1}$ to $n^{-4/5}$, depending on the smoothness properties of the PDF of $X$, denoted by $p$.

My question: Is there an equivalent result in the $L^1$ norm/ total variation. In other words, what can be said on $E_{X_1, \ldots ,X_n} \left[\|\hat{p}_{\rm kde} - p\|_1 \right] $? What is the asymptotic optimal rate of its convergence?

Total variation and $L^1$ : Although this is not the standard definition of TV, it is well known that for two densities $p$ and $q$, $\frac{1}{2}\|p-q \|_{\rm TV} = \|p-q\|_1 :\,= \int\limits_{\mathbb{R}} |p(y)-q(y)| \, dy $. See e.g, here

Motivation: In simulations, I observe that the asymptotic $L^1$ distance is $\approx n^{-0.3}$, but I'm not sure whether that an implementation problem or the actual result.

$\endgroup$
1
$\begingroup$

The short answer, broadly speaking, is that if the PDF $p$ is $s$ times differentiable, the lower bound on the expectency of $\|p-\hat{p}_{\rm kde} \|_1$ is $n^{-\frac{s}{2s+1}}$, which means that for an analytic PDF, we approach $n^{-\frac{1}{2}}$.

I owe this answer to the wonderful textbook - "Nonparametric Density Estimation - The L1 View" by Devroye and Gyorfi, which was made online by the authors here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.