In (univariate) kernel density estimation, I often come across constructions where some Taylor expansion like

$ \int K_h ( u - y) f(y) dy = \int K(x) f( u - hx) dx = \sum_{k = 0}^2 h^{2k} f^{(2k)} (u) \frac{m_{2k}(K)}{(2k)!} + O(h^6)$

is done and then (may the following $O(h^6)$ represent the same term as above)

$ \int \int L_g(z) O(h^6) f(u + hz) dz du = m_0(L)O(h^6) + m_2(L) O(g^2 h^6) + O(g^4 h^6), $

where $L, K$ are bounded symmetric kernels with finite moments $m_1, ... m_8$ and bandwidths $h,g$. Furthermore $ \text{sup}_{x \in \mathbb R} | f^{(j)}(x) < \infty $ for $j = 1, ..., 6$. My question is, why the $O(h^6)$ remains under integration? Couldn't it be possible that even if the kernels are bounded, they have some infinite higher moments or that terms like $\int f^{(k)} O(h^6)$ for big $k$ are infinite? Or do I need some further conditions?

  • $\begingroup$ do you have a reference for where your issue occurs? $\endgroup$ Jun 1, 2016 at 12:40

1 Answer 1


I am not entirely sure where (the need for) the double integrals comes from (maybe you can elaborate), but anyway, here is some hopefully related information:

The first line seems to be related to the derivation of the expected value of the KDE, reproduced below:

\begin{align*} E\left[ \widehat{f}(x)\right] =& \int k(\psi )f(x+h\psi )d\psi \\ = & \int \left( f(x)+(h\psi )f^{(1)}(x)+\frac{(h\psi )^{2}}{2}% f^{(2)}(x)+O(h^{3})\right) k(\psi )d\psi \\ =& f(x)\int k(\psi )d\psi +hf^{(1)}(x)\int \psi k(\psi )d\psi \\ &+\frac{h^{2}}{2}f^{(2)}(x)\int \psi ^{2}k(\psi )d\psi +O(h^{3}) \\ =& f(x)\kappa _{0}(k)+hf^{(1)}(x)\kappa _{1}(k)+\frac{h^{2}}{2}% f^{(2)}(x)\kappa _{2}(k)+O(h^{3}) \\ =& f(x)+\frac{h^{2}}{2}f^{(2)}(x)\kappa _{2}(k)+O(h^{3}), \end{align*} where the $O(h^3)$ term comes from bounding the final term in the integral over the Taylor expansion of $f$. $$ (1/3!)h^3\left|\int f^{(3)}(\tilde x)\psi^3k(\psi)d\psi\right|\leq Ch^3\int\left|\psi^3k(\psi)\right|d\psi=O(h^3), $$ where $C$ is some positive constant and $\tilde x$ lies between $x$ and $x+h\psi$. For this to work, $f$ must be three times differentiable. Moreover, a bounded kernel $k$ ensures that $\int\left|\psi^3k(\psi)\right|d\psi<\infty$.

  • $\begingroup$ Thank you! The second equation refers (slightly modified for simplicity) to the article "A simple root n bandwidth selector" by Jones, Marron, Park [1991], proof of lemma 1. The kind of problem I ment appears for example in $\int bias^2( \hat{f} (x),f(x)) dx$ where we have a term like $\int f''(x) o(h^2) dx$. If I assume that $f''$ is ultimately monotone, I can proof, it is finite, but such condition was not given in that article (or elsewhere), so maybe there is another way? $\endgroup$
    – xxx
    Jun 2, 2016 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.