Kernel density estimation: Order remains under integration

Question

In (univariate) kernel density estimation, I often come across constructions where some Taylor expansion like

$ \int K_h ( u - y) f(y) dy = \int K(x) f( u - hx) dx = \sum_{k = 0}^2 h^{2k} f^{(2k)} (u) \frac{m_{2k}(K)}{(2k)!} + O(h^6)$

is done and then (may the following $O(h^6)$ represent the same term as above)

$ \int \int L_g(z) O(h^6) f(u + hz) dz du = m_0(L)O(h^6) + m_2(L) O(g^2 h^6) + O(g^4 h^6), $

where $L, K$ are bounded symmetric kernels with finite moments $m_1, ... m_8$ and bandwidths $h,g$. Furthermore $ \text{sup}_{x \in \mathbb R} | f^{(j)}(x) < \infty $ for $j = 1, ..., 6$. My question is, why the $O(h^6)$ remains under integration? Couldn't it be possible that even if the kernels are bounded, they have some infinite higher moments or that terms like $\int f^{(k)} O(h^6)$ for big $k$ are infinite? Or do I need some further conditions?

Answer 1

I am not entirely sure where (the need for) the double integrals comes from (maybe you can elaborate), but anyway, here is some hopefully related information:

The first line seems to be related to the derivation of the expected value of the KDE, reproduced below:

\begin{align*} E\left[ \widehat{f}(x)\right] =& \int k(\psi )f(x+h\psi )d\psi \\ = & \int \left( f(x)+(h\psi )f^{(1)}(x)+\frac{(h\psi )^{2}}{2}% f^{(2)}(x)+O(h^{3})\right) k(\psi )d\psi \\ =& f(x)\int k(\psi )d\psi +hf^{(1)}(x)\int \psi k(\psi )d\psi \\ &+\frac{h^{2}}{2}f^{(2)}(x)\int \psi ^{2}k(\psi )d\psi +O(h^{3}) \\ =& f(x)\kappa _{0}(k)+hf^{(1)}(x)\kappa _{1}(k)+\frac{h^{2}}{2}% f^{(2)}(x)\kappa _{2}(k)+O(h^{3}) \\ =& f(x)+\frac{h^{2}}{2}f^{(2)}(x)\kappa _{2}(k)+O(h^{3}), \end{align*} where the $O(h^3)$ term comes from bounding the final term in the integral over the Taylor expansion of $f$. $$ (1/3!)h^3\left|\int f^{(3)}(\tilde x)\psi^3k(\psi)d\psi\right|\leq Ch^3\int\left|\psi^3k(\psi)\right|d\psi=O(h^3), $$ where $C$ is some positive constant and $\tilde x$ lies between $x$ and $x+h\psi$. For this to work, $f$ must be three times differentiable. Moreover, a bounded kernel $k$ ensures that $\int\left|\psi^3k(\psi)\right|d\psi<\infty$.