2
$\begingroup$

My question refers to the book "Nonparametric Econometrics - Theory and Practice" by Li & Racine. Here, the variance for a kernel density estimator using the pointwise perspective (for fixed x) is derived as followed: \begin{align} var(\hat{f}_n(x))&=var\Big(\frac{1}{nh}\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\ &=\frac{1}{n^2h^2}var\Big(\sum^n_{i=1}k(\frac{X_i-x}{h})\Big)\\ &=\frac{1}{nh^2}var\Big(k(\frac{X_1-x}{h})\Big)\\ &=\frac{1}{nh^2}(E(k(\frac{X_i-x}{h})^2)-E(k(\frac{X_1-x}{h}))^2)\\ &=\frac{1}{nh^2}\Big(h\int f(x+h*u) k^2(u)du-(h\int f(x+hu)*k(u)du)^2\Big)\\ &=\frac{1}{nh^2}\Big(h\int (f(x)+f^{(1)}(x)hu) k^2(u)du-O(h^2)\Big)\\ &=\frac{1}{nh}\Big(f(x)\int k^2(u)du+O(h\int|u|k^2(u)du)-O(h)\Big)\\ &=\frac{1}{nh}(\kappa f(x)+O(h)) \end{align} , here k is a kernel function with classical assumptions, $X_i,x_1$ realizations, f the true density, h a bandwidth and n the sample size, besides $\kappa=\int k^2(u)du$. What I cannot understand are the last three equalities, i.e. why $\int f^{(1)}(x)hu*k^2(u)du$ results in the bounded term with $O(h\int|u|k^2(u)du)$. The boundedness is obvious since the first derivative is some constant at given x.

  • How does one obtain the particular value for the Big O upper bound (especially in the form where the absolute value of u is used)?
  • And how are the two Big O terms subtracted from each other to obtain the final equality with O(h)?

I appreciate any help!

Improve this question
$\endgroup$
1
  • $\begingroup$ If $ \int_{\mathbb{R}}k^2(u)u\mathrm{d}u$ is assumed to be finite, then $O(h\int|u|k^2(u)du)$ should follow from the triangle inequality for integrals, right? In a textbook of mine, $\mathrm{Var}[k(X)]$ is assumed to be finite, which would imply $ \int_{\mathbb{R}}k^2(x)f(x)\mathrm{d}x$ to be finite, where $f$ is the density of $X$. But I'm unsure if this would also imply $ \int_{\mathbb{R}}k^2(u)u\mathrm{d}u$ to be finite. $\endgroup$
    – psie
    Jul 1, 2021 at 11:13

1 Answer 1

Reset to default
0
$\begingroup$

The $O(\cdot)$ notation can hide away constants and is a slight abuse of notation (you will see people occasionally use set notation instead). Note that for an expression, $a(h)$, we say it is $O(h)$ if there exists a constant $C>0$ such that: $$ |a(h)| \leq C \cdot h \text{ for all } h > 0$$ There are more precise definitions, but this suffices for what you need to prove in this exercise.

So now let us check, what does it mean to say $O(h) - O(h) = O(h)$? The LHS $O(h)$ terms correspond to a specific expression that got summarized earlier on, let us call it $a_1(h) = O(h)$ and the second one $a_2(h) = O(h)$. So we need to show that $a_1(h) - a_2(h) = O(h)$. Take $C_1,C_2$ such that $|a_1(h)| \leq C_1 h$ and $|a_2(h)| \leq C_2 h$ , then:

$$ |a_1(h) - a_2(h)| \leq |a_1(h) + a_2(h)| \leq (C_1 + C_2)h \text { for all }h > 0 $$

So indeed $a_1(h) - a_2(h) = O(h)$, i.e., $O(h) - O(h) = O(h)$. Pause a moment to note that the LHS has a different meaning in this expression than the RHS.

Similarly $a(h) = O(h^2)$ means that:

$$ |a(h)| \leq C \cdot h^2 \text{ for all } h > 0$$

One final remark: Li & Racine derive asymptotics for small $h>0$, so that in all of the above expressions it suffices to check that the $O(\cdot)$ conditions hold for small $h$, say for $0 < h < H$ where $H$ is a small constant.

Can you verify that:

  • $O(h^2) = O(h)$?
  • The other questions you asked?
Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks a lot, that makes sense! $O(h^2)$ is therefore $O(h)$ since there is some $C_2>0$ s.t. $|a(h)|\leq C_1*h^2\leq C_2 *h$. Can you elaborate why the integral can be written as $\int|u|k^2(u)du$? The indefinite integral should be zero s.t. the term vanishes by the kernel properties, however using the absolute $|u|$ this will not be the case. $\endgroup$
    – Henry
    May 18, 2020 at 7:47
  • 1
    $\begingroup$ My guess, since the term is symmetric, and $O(h*\int^\infty_{0}vk^2(u)du)+O(h*\int^0_{-\infty}vk^2(u)du))=O(h)$, you simply rewrite the term using the absolutes since the $O(h)$ notation omits the constants anyways? $\endgroup$
    – Henry
    May 18, 2020 at 8:02
  • $\begingroup$ Yes! You will be treating this part as a constant, so it's just an intermediate step putting in the absolute value inside (since I guess the assumptions will have been stated in terms of that quantity). $\endgroup$
    – air
    May 18, 2020 at 8:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.