Expectation of derivative of kernel density estimator

Question

I am trying to calculate the expectation of the $s$'th derivative of a kernel density estimator. This problem arises naturally when trying to estimate the derivative of a density, because one approach - natural to some - is to define the estimator of the derivative of the density as the derivative of the estimator of the density. To put this formally let $\hat f(x)$ be the kernel density estimator then

$$\hat f^s(x) := \frac{d}{dx^s} \hat f(x)$$

In the case where the kernel density estimator is defined as $$ \hat f(x) := \frac{1}{nh} \sum_i K\left(\frac{x_i-x}{h} \right)$$ the implication is that

$$\hat f^s(x) := \frac{d^s}{dx^s} \hat f(x) = \frac{(-1)^s}{nh^{s+1}} \sum_i K^s\left(\frac{x_i - x}{h} \right)$$ where $K^s(u) = \frac{d^s}{du^s} K(u)$, as mentioned in for example (Ullah & Pagan 1999) equation (2.28) and easily verified.

In a note it is mentioned that by integration by parts, it can be shown that

$$ \mathbb E[\hat f^s(x)] = \int_{-\infty}^\infty \frac{1}{h} K^s\left(\frac{t - x}{h} \right)f(t)dt$$

Trying to convince myself that this is correct I start by removing the factor (1/n) from the expectation:

$$ \mathbb E[\hat f^s(x)] = \frac{(-1)^s}{nh^{s+1}} \sum_i^n \mathbb E\left[K^s\left(\frac{X_i -x}{h} \right) \right] = \frac{(-1)^s}{h^{s+1}} \mathbb E\left[K^s\left(\frac{X_i - x}{h} \right) \right] $$ since the expectation is taken with respect to the sample $X_1,...,X_n$ assumed iid. Let $X_i \sim f(t)$ such that

$$ \mathbb E[\hat f^s(x)] = \frac{(-1)^s}{h^{s+1}} \int K^s\left(\frac{t-x}{h} \right) f(t) dt$$

so far so good. Now comes application of integration by parts, $wv' = wv - \int vw'$, where the idea is to integrate $K^s$ and differentiate $f$, hence let $w=f$ and $v' = \frac{1}{h}K^s$. So we use one factor (1/h) in combination with $K^s$ to get

$$\frac{(-1)^s}{h^{s}} \int_a^b \frac{1}{h}K^s\left(\frac{t-x}{h} \right) f(t) dt = \frac{(-1)^s}{h^{s}} \left[ \left[K^{s-1}\left(\frac{x - t}{h} \right) f(t)\right]_a^b - \int_a^b K^{s-1}\left(\frac{x - t}{h} \right) f^1(t)dt \right]$$ my guess is that by some properties of the kernels and or density it follows that $$\left[K^{s-1}\left(\frac{x - t}{h} \right) f(t)\right]_a^b = 0$$ and perhaps all that is needed for this to be the case when the limits are $\infty$ and $-\infty$ is simply that

$$\lim_{t\rightarrow \infty}K^{s-1}\left(\frac{x - t}{h} \right) f(t) = 0$$ and $$\lim_{t\rightarrow -\infty}K^{s-1}\left(\frac{x - t}{h} \right) f(t) = 0$$

but I'm kind of stuck so my question is how to continue?

Answer 1

Your intuition on the limit is correct (cv.this Math.SE post) provided $K$ is bounded, which it is if it’s a kernel.

Then I guess you are trying to compute the expectation to verify whether it is a biased estimator, which means that you are going to end up with an expression such as

$$h^{-1} \int_{-\infty}^{+\infty} K\left(\frac{t-x}{h}\right)f’(t)dt - f’(x)$$

Then you can make a change of variables

$$\int_{-\infty}^{+\infty} K(u)f’(x + hu)du - f’(x)$$

And take a Taylor series of

$$f’(x + hu)= f’(x) + \ldots + f^{(l)} (x + \theta u h) \frac{(uh)^{l-1}}{(l-1)!} $$ for some $\theta \in [0, 1]$.

The first term will cancel out with the rightmost term just outside of the integral, all the other terms except for the last one will be zeroed out since the kernel is of order $l$, and then see where you go from there :)