# Derivation of variance for kernel density estimator

My question refers to the book "Nonparametric Econometrics - Theory and Practice" by Li & Racine. Here, the variance for a kernel density estimator using the pointwise perspective (for fixed x) is derived as followed: \begin{align} var(\hat{f}_n(x))&=var\Big(\frac{1}{nh}\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\ &=\frac{1}{n^2h^2}var\Big(\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\ &=\frac{1}{nh^2}var\Big(k(\frac{X_1-x}{h})\Big)\\ &=\frac{1}{nh^2}(E(k(\frac{X_i-x}{h})^2)-E(k(\frac{X_1-x}{h}))^2)\\ &=\frac{1}{nh^2}\Big(h\int f(x+h*u) k^2(u)du-(h\int f(x+hu)*k(u)du)^2\Big)\\ &=\frac{1}{nh^2}\Big(h\int (f(x)+f^{(1)}(x)hu) k^2(u)du-O(h^2)\Big)\\ &=\frac{1}{nh}\Big(f(x)\int k^2(u)du+O(h\int|u|k^2(u)du)-O(h)\Big)\\ &=\frac{1}{nh}(\kappa f(x)+O(h)) \end{align} , here k is a kernel function with classical assumptions, $$X_i,x_1$$ realizations, f the true density, h a bandwidth and n the sample size, besides $$\kappa=\int k^2(u)du$$. What I cannot understand are the last three equalities, i.e. why $$\int f^{(1)}(x)hu*k^2(u)du$$ results in the bounded term with $$O(h\int|u|k^2(u)du)$$. The boundedness is obvious since the first derivative is some constant at given x.

• How does one obtain the particular value for the Big O upper bound (especially in the form where the absolute value of u is used)?
• And how are the two Big O terms subtracted from each other to obtain the final equality with O(h)?

I appreciate any help!

• If $\int_{\mathbb{R}}k^2(u)u\mathrm{d}u$ is assumed to be finite, then $O(h\int|u|k^2(u)du)$ should follow from the triangle inequality for integrals, right? In a textbook of mine, $\mathrm{Var}[k(X)]$ is assumed to be finite, which would imply $\int_{\mathbb{R}}k^2(x)f(x)\mathrm{d}x$ to be finite, where $f$ is the density of $X$. But I'm unsure if this would also imply $\int_{\mathbb{R}}k^2(u)u\mathrm{d}u$ to be finite.
– schn
Jul 1, 2021 at 11:13

## 1 Answer

The $$O(\cdot)$$ notation can hide away constants and is a slight abuse of notation (you will see people occasionally use set notation instead). Note that for an expression, $$a(h)$$, we say it is $$O(h)$$ if there exists a constant $$C>0$$ such that: $$|a(h)| \leq C \cdot h \text{ for all } h > 0$$ There are more precise definitions, but this suffices for what you need to prove in this exercise.

So now let us check, what does it mean to say $$O(h) - O(h) = O(h)$$? The LHS $$O(h)$$ terms correspond to a specific expression that got summarized earlier on, let us call it $$a_1(h) = O(h)$$ and the second one $$a_2(h) = O(h)$$. So we need to show that $$a_1(h) - a_2(h) = O(h)$$. Take $$C_1,C_2$$ such that $$|a_1(h)| \leq C_1 h$$ and $$|a_2(h)| \leq C_2 h$$ , then:

$$|a_1(h) - a_2(h)| \leq |a_1(h) + a_2(h)| \leq (C_1 + C_2)h \text { for all }h > 0$$

So indeed $$a_1(h) - a_2(h) = O(h)$$, i.e., $$O(h) - O(h) = O(h)$$. Pause a moment to note that the LHS has a different meaning in this expression than the RHS.

Similarly $$a(h) = O(h^2)$$ means that:

$$|a(h)| \leq C \cdot h^2 \text{ for all } h > 0$$

One final remark: Li & Racine derive asymptotics for small $$h>0$$, so that in all of the above expressions it suffices to check that the $$O(\cdot)$$ conditions hold for small $$h$$, say for $$0 < h < H$$ where $$H$$ is a small constant.

Can you verify that:

• $$O(h^2) = O(h)$$?
• The other questions you asked?
• Thanks a lot, that makes sense! $O(h^2)$ is therefore $O(h)$ since there is some $C_2>0$ s.t. $|a(h)|\leq C_1*h^2\leq C_2 *h$. Can you elaborate why the integral can be written as $\int|u|k^2(u)du$? The indefinite integral should be zero s.t. the term vanishes by the kernel properties, however using the absolute $|u|$ this will not be the case. May 18, 2020 at 7:47
• My guess, since the term is symmetric, and $O(h*\int^\infty_{0}vk^2(u)du)+O(h*\int^0_{-\infty}vk^2(u)du))=O(h)$, you simply rewrite the term using the absolutes since the $O(h)$ notation omits the constants anyways? May 18, 2020 at 8:02
• Yes! You will be treating this part as a constant, so it's just an intermediate step putting in the absolute value inside (since I guess the assumptions will have been stated in terms of that quantity).
– air
May 18, 2020 at 8:29