# Proving that the bias of the derivative of Parzen-Rosenblatt (kernel density) estimator is of order $O(h^2)$ and $O(h)$ when $h$ approaches $0$

I came across this property that I don't get and I couldn't find the proof anywhere:

Suppose we have a density $$K$$ of the standard normal distribution and $$K'$$ its derivative. Suppose that the density $$f$$ is of class $$C^4$$ in $$\Bbb{R}$$ .

Let $$\hat{f'}_{h,n} = \cfrac{1}{nh^2}\sum_{j=1}^n K'(\cfrac{x-X_j}{h})$$ be the estimator of the density $$f'$$ with $$h > 0$$.

For a small $$h$$, as in if $$h$$ approaches $$0$$ the bias verifies :

$$B(\hat{f'}_{h,n}) = O(h)$$ and $$B(\hat{f'}_{h,n}) = O(h^2)$$.

Can someone explain to me how did we get to that result?

EDIT: I found this theorem:

But here we are discussing the density $$f$$ and not $$f'$$. How can we extend the proposition?

EDIT: Here is my attempt :

$$E(\hat{f'}_{h,n}(x)) = \int\cfrac{1}{h^2}K'(\cfrac{u-x}{h})f'(u)$$

Let $$u = x +hv$$ so we get : $$E(\hat{f'}_{h,n}(x)) = \cfrac{1}{h}\int f'(hv+x) K'(v)dv$$

Now I'm thinking about using taylor's expansion for f because when $$h$$ approaches 0:

$$f'(hv+x) = f^{(2)}hv +\cfrac{(hv)^2}{2} f^{3}(x)+\cfrac{(hv)^3}{3!}f^{4}(x)+ o(h^3)$$ but seeing this calculation, I don't know whether it will get me to prove that the order is indeed $$O(h^2)$$ and $$O(h)$$.

How should I proceed?

• 1, Why do you refer to the order as being both $O(h^2)$ and $O(h)$? 2. $O(h^2)$ is not the same as $o(h^2)$. Commented Feb 28, 2023 at 21:32
• @jbowman thanks for your reply. 1/ In the proposition I came across both $O(h^2)$ and $O(h)$ which is part of my confusion. 2/ I know that $O(h^2)$ and $o(h^2)$ aren't the same but $o(h)$ implies $O(h)$ so I thought taylor expansion was a good start. Commented Feb 28, 2023 at 21:51
• You have to use the Taylor series expansion and the calculations are not very good looking (especially near the boundary.) Just after a quick googling, I found this link: faculty.washington.edu/yenchic/17Sp_403/Lec7-density.pdf Do you want me to give this as an answer? Commented Mar 1, 2023 at 6:17
• @SubrataPal which part of the calculation I did is wrong? Is taking $f'$ inside of the integral is wrong? I'd appreciate your answer yes Commented Mar 1, 2023 at 14:18
• @wageeh No. Sorry, I misread the question the first time; I am attaching my answer. Commented Mar 2, 2023 at 2:06

With some abuse in notation, we can write, for any fixed $$x,$$ \begin{align} h E(\hat{f}'(x)) &= \frac{1}{h} \mathbb E\left[K'\left(\frac{X-x}{h} \right) \right]\\ &=\int f(t) \cdot \frac{1}{h} K'\left(\frac{t-x}{h} \right) dt \\ &=\Bigg[f(t) \cdot K\left(\frac{t-x}{h}\right)\Bigg]_{-\infty}^{\infty} - \int K\left(\frac{t-x}{h} \right) \cdot f'(t)dt \\ &= -\int K\left(\frac{t-x}{h} \right) \cdot f'(t)dt \end{align} using integration by parts, and both $$f$$ and $$K$$ are valid pdf.
Now, take $$u=(t-x)/h$$ and $$du=dt/h,$$ gives \begin{align} E(\hat{f}'(x)) - f'(x) &= \frac{-1}{h}\int K\left(\frac{t-x}{h} \right) \cdot f'(t)dt - f'(x)\\ &= -\int K(u) \cdot \left[f'(x+uh)-f'(x)\right] du \\ &= -\int K(u) \cdot \left[f''(x)(uh)+f'''(x)(uh)^2/2 + o(h^2)\right] du \end{align} which you can check is $$O(h)$$.
I think derivative in both $$K$$ and $$f$$ is not possible in your approach.