My question refers to the book "Nonparametric Econometrics - Theory and Practice" by Li & Racine. Here, the variance for a kernel density estimator using the pointwise perspective (for fixed x) is derived as followed: \begin{align} var(\hat{f}_n(x))&=var\Big(\frac{1}{nh}\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\ &=\frac{1}{n^2h^2}var\Big(\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\ &=\frac{1}{nh^2}var\Big(k(\frac{X_1-x}{h})\Big)\\ &=\frac{1}{nh^2}(E(k(\frac{X_i-x}{h})^2)-E(k(\frac{X_1-x}{h}))^2)\\ &=\frac{1}{nh^2}\Big(h\int f(x+h*u) k^2(u)du-(h\int f(x+hu)*k(u)du)^2\Big)\\ &=\frac{1}{nh^2}\Big(h\int (f(x)+f^{(1)}(x)hu) k^2(u)du-O(h^2)\Big)\\ &=\frac{1}{nh}\Big(f(x)\int k^2(u)du+O(h\int|u|k^2(u)du)-O(h)\Big)\\ &=\frac{1}{nh}(\kappa f(x)+O(h)) \end{align} , here k is a kernel function with classical assumptions, $X_i,x_1$ realizations, f the true density, h a bandwidth and n the sample size, besides $\kappa=\int k^2(u)du$. What I cannot understand are the last three equalities, i.e. why $\int f^{(1)}(x)hu*k^2(u)du$ results in the bounded term with $O(h\int|u|k^2(u)du)$. The boundedness is obvious since the first derivative is some constant at given x.

  • How does one obtain the particular value for the Big O upper bound (especially in the form where the absolute value of u is used)?
  • And how are the two Big O terms subtracted from each other to obtain the final equality with O(h)?

I appreciate any help!


The $O(\cdot)$ notation can hide away constants and is a slight abuse of notation (you will see people occasionally use set notation instead). Note that for an expression, $a(h)$, we say it is $O(h)$ if there exists a constant $C>0$ such that: $$ |a(h)| \leq C \cdot h \text{ for all } h > 0$$ There are more precise definitions, but this suffices for what you need to prove in this exercise.

So now let us check, what does it mean to say $O(h) - O(h) = O(h)$? The LHS $O(h)$ terms correspond to a specific expression that got summarized earlier on, let us call it $a_1(h) = O(h)$ and the second one $a_2(h) = O(h)$. So we need to show that $a_1(h) - a_2(h) = O(h)$. Take $C_1,C_2$ such that $|a_1(h)| \leq C_1 h$ and $|a_2(h)| \leq C_2 h$ , then:

$$ |a_1(h) - a_2(h)| \leq |a_1(h) + a_2(h)| \leq (C_1 + C_2)h \text { for all }h > 0 $$

So indeed $a_1(h) - a_2(h) = O(h)$, i.e., $O(h) - O(h) = O(h)$. Pause a moment to note that the LHS has a different meaning in this expression than the RHS.

Similarly $a(h) = O(h^2)$ means that:

$$ |a(h)| \leq C \cdot h^2 \text{ for all } h > 0$$

One final remark: Li & Racine derive asymptotics for small $h>0$, so that in all of the above expressions it suffices to check that the $O(\cdot)$ conditions hold for small $h$, say for $0 < h < H$ where $H$ is a small constant.

Can you verify that:

  • $O(h^2) = O(h)$?
  • The other questions you asked?
| cite | improve this answer | |
  • $\begingroup$ Thanks a lot, that makes sense! $O(h^2)$ is therefore $O(h)$ since there is some $C_2>0$ s.t. $|a(h)|\leq C_1*h^2\leq C_2 *h$. Can you elaborate why the integral can be written as $\int|u|k^2(u)du$? The indefinite integral should be zero s.t. the term vanishes by the kernel properties, however using the absolute $|u|$ this will not be the case. $\endgroup$ – Henry May 18 at 7:47
  • 1
    $\begingroup$ My guess, since the term is symmetric, and $O(h*\int^\infty_{0}vk^2(u)du)+O(h*\int^0_{-\infty}vk^2(u)du))=O(h)$, you simply rewrite the term using the absolutes since the $O(h)$ notation omits the constants anyways? $\endgroup$ – Henry May 18 at 8:02
  • $\begingroup$ Yes! You will be treating this part as a constant, so it's just an intermediate step putting in the absolute value inside (since I guess the assumptions will have been stated in terms of that quantity). $\endgroup$ – air May 18 at 8:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.