Derivation of variance for kernel density estimator My question refers to the book "Nonparametric Econometrics - Theory and Practice" by Li & Racine. Here, the variance for a kernel density estimator using the pointwise perspective (for fixed x) is derived as followed:
\begin{align}
var(\hat{f}_n(x))&=var\Big(\frac{1}{nh}\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\
&=\frac{1}{n^2h^2}var\Big(\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\
&=\frac{1}{nh^2}var\Big(k(\frac{X_1-x}{h})\Big)\\
&=\frac{1}{nh^2}(E(k(\frac{X_i-x}{h})^2)-E(k(\frac{X_1-x}{h}))^2)\\
&=\frac{1}{nh^2}\Big(h\int f(x+h*u) k^2(u)du-(h\int f(x+hu)*k(u)du)^2\Big)\\
&=\frac{1}{nh^2}\Big(h\int (f(x)+f^{(1)}(x)hu) k^2(u)du-O(h^2)\Big)\\
&=\frac{1}{nh}\Big(f(x)\int k^2(u)du+O(h\int|u|k^2(u)du)-O(h)\Big)\\
&=\frac{1}{nh}(\kappa f(x)+O(h)) 
\end{align}
, here k is a kernel function with classical assumptions, $X_i,x_1$ realizations, f the true density, h a bandwidth and n the sample size, besides $\kappa=\int k^2(u)du$. 
What I cannot understand are the last three equalities, i.e. why $\int f^{(1)}(x)hu*k^2(u)du$ results in the bounded term with $O(h\int|u|k^2(u)du)$. The boundedness is obvious since the first derivative is some constant at given x. 


*

*How does one obtain the particular value for the Big O upper bound (especially in the form where the absolute value of u is used)?

*And how are the two Big O terms subtracted from each other to obtain the final equality with O(h)?


I appreciate any help!
 A: The $O(\cdot)$ notation can hide away constants and is a slight abuse of notation (you will see people occasionally use set notation instead). Note that for an expression, $a(h)$, we say it is $O(h)$ if there exists a constant $C>0$ such that:
$$ |a(h)| \leq  C \cdot h  \text{ for all } h > 0$$ 
There are more precise definitions, but this suffices for what you need to prove in this exercise. 
So now let us check, what does it mean to say $O(h) - O(h) = O(h)$? The LHS $O(h)$ terms correspond to a specific expression that got summarized earlier on, let us call it $a_1(h) = O(h)$ and the second one $a_2(h) = O(h)$. So we need to show that $a_1(h) - a_2(h) = O(h)$. Take $C_1,C_2$ such that $|a_1(h)| \leq C_1 h$ and $|a_2(h)| \leq C_2 h$ , then:
$$ |a_1(h) - a_2(h)| \leq |a_1(h) + a_2(h)| \leq (C_1 + C_2)h \text { for all }h > 0 $$
So indeed $a_1(h) - a_2(h) = O(h)$, i.e., $O(h) - O(h) = O(h)$. Pause a moment to note that the LHS has a different meaning in this expression than the RHS.
Similarly $a(h) = O(h^2)$ means that:
$$ |a(h)| \leq  C \cdot h^2  \text{ for all } h > 0$$ 
One final remark: Li & Racine derive asymptotics for small $h>0$, so that in all of the above expressions it suffices to check that the $O(\cdot)$ conditions hold for small $h$, say for $0 < h < H$ where $H$ is a small constant. 
Can you verify that: 


*

*$O(h^2) = O(h)$?

*The other questions you asked?

