# Convergence of kernel density estimate as the sample size grows

Let $$X\sim\text{Normal}(0,1)$$ and let $$f_X$$ be its probability density function. I conducted some numerical experiments in the software Mathematica to estimate $$f_X$$ via a kernel method. Let $$\hat{f}_X^M$$ be the kernel density estimate using a sample of length $$M$$. Let $$\epsilon=E\left[\|f_X-\hat{f}_X^M\|_\infty\right]$$ be the error ($$E$$ is the expectation). I noticed that the error decreases with $$M$$ until a certain length $$M_0$$ from which the error stabilizes. For example, in Mathematica, the built-in function SmoothKernelDistribution employs the Gaussian kernel with Silverman's rule to determine the bandwidth by default. In the following figure in log-log scale, I show the error $$\epsilon$$ for different values of $$M$$ growing geometrically, where the expectation that defines $$\epsilon$$ is estimated using 20 realizations of $$\|f_X-\hat{f}_X^M\|_\infty$$. I also plot the estimated $$90\%$$ confidence interval for $$\|f_X-\hat{f}_X^M\|_\infty$$ (dashed lines).

Observe that the error decreases linearly in log-log scale (that is, at rate $$O(M^{-r})$$), up to a certain length $$M$$ where it starts to stabilize. Also, the confidence intervals become more narrow in the end. Is this issue due to accumulated numerical errors? Is it due to Silverman's rule?

I think the problem is the built-in function of Mathematica, SmoothKernelDistribution (its interpretation of Silverman's selection). If you implement the estimator $$\hat{f}_X^M(x)=\frac{1}{Mh}\sum_{i=1}^M K\left(\frac{x-x_i}{h}\right)$$ yourself, where $$x_1,\ldots,x_M$$ is the data, the kernel $$K$$ is the density function of a $$\text{Normal}(0,1)$$ distribution, and the bandwidth $$h$$ is $$1.06\,\hat{\sigma}_X\,M^{-1/5}$$, then the error tends to zero as $$M$$ grows with no problem:

The bandwidth selected by SmoothKernelDistribution seems to be too large when $$M$$ gets bigger, which implies a biased estimate (error stabilization) with small variance (more narrow confidence interval).

Any other response will be welcomed.