# Properties of Kernel Density Estimators

## Given

Let $$X \in \mathbb{R}$$ be a real-valued random variable with theoretical probability density function (pdf) $$f(x)$$ and corresponding cumulative distribution function (cdf) $$F(x)$$. Let $$X_1, X_2, \cdots, X_n$$ a random sample of size $$n$$ drawn according to the distribution of $$X$$. Let $$h>0$$ be a positive real number and consider the kernel density estimator $$\widehat{f}_n$$ of $$f$$ given for every $$x \in \mathcal{X}$$ by $$\begin{eqnarray} \label{eq:kde:1d:1} \widehat{f}_n(x;h) = \frac{1}{nh}\sum_{i=1}^n{K\left(\frac{x-X_i}{h}\right)} \end{eqnarray}$$ where the kernel in this case is the so-called boxcar kernel defined by $$\begin{eqnarray} \label{eq:kernel:boxcar} K(u) = \left\{\begin{array}{ll} 1 & -\frac{1}{2} < u < \frac{1}{2}\\ 0 & \texttt{Otherwise}. \end{array} \right. \end{eqnarray}$$

## Questions

1. Prove that the boxcar kernel of $$K(u)$$ is a bona fide kernel.
2. Show that $$\mathbb{E}(\widehat{f}_n(x; h)) = \frac{1}{h}\displaystyle \int_{x-(h/2)}^{x+(h/2)}{f(v)dv}$$
3. Show that $$\mathbb{V}(\widehat{f}_n(x; h)) = \frac{1}{nh^2}\left[\int_{x-(h/2)}^{x+(h/2)}{f(v)dv}-\left(\int_{x-(h/2)}^{x+(h/2)}{f(v)dv}\right)^2\right]$$

## Attempts

1. A bona fide kernel is one that is real, genuine, legitimate. i.e. a density estimation function that is non-negative and integrates to $$1$$. How do I show that a function is non-negative? I know that there is a relationship between the Bias of a density estimation function and its bona fide-ness, but I'm not sure what that relationship is. Any direction here would be helpful.

2. I know this by definition: $$\mathbb{E}(\hat{f}(x)) = \int \frac{1}{h} K \big( \frac{x-y}{h} \big) f(y)dy$$ So I can use that formula, but what is $$h$$, and how do I substitute $$f(v)dv$$ in place of $$f(y)dy$$?

3. Similarly, I know this by definition: $$\mathbb{V}(\hat{f}(x)) = \int \frac{1}{h^2} K \big( \frac{x-y}{h} \big)^2 f(y)dy - \Big( \frac{1}{h} \int K \big(\ \frac{x-y}{h} \big) f(y)dy \Big)^2$$ But I still have the same confusions: but what is $$h$$, and how do I substitute $$f(v)dv$$ in place of $$f(y)dy$$?

Thank you in advance for any help/clarification you can provide!

• It's unclear where you're trying to go with "How do I show that a function is non-negative?" In this case, the values of $K$ are explicitly limited to $0$ and $1$: what more is there to say? You answer your second question, "what is $h,$" earlier in the text where you write "Let $h\gt 0$ be a positive real number." The third is equally trivial: the integral does not change when you rename the variable "$y.$" – whuber Mar 13 '19 at 14:54
• (1) That makes sense, thank you! My calculus isn't great: how would I compute $\int K(u)du$? (2) Yes, $h$ is defined to be a positive real number, but what does $h$ represent? What is its purpose? (3) So, I just need to define $f(v) = K \big( \frac{x-y}{h} \big)^2 f(y)dy$ and substitute? – inkalchemist1994 Mar 13 '19 at 15:00
• (1) Draw a picture: the integral asks you to compute the area of a rectangle. (2) $h$ is the bandwidth. Again, draw pictures for different values of $h:$ then you will see what it represents. (3) If you're unsure, you definitely need to review the basics of integral calculus. Just the basics--don't worry about the computational techniques. Once you understand what the symbols mean, you will progress much more quickly. – whuber Mar 13 '19 at 15:04
• (2) If I start from $\mathbb{E}[\hat{f}_n(x; h)] = \frac{1}{h}\mathbb{E}[K(\frac{x-X_i}{h})]$, then I get $= \frac{1}{h} \int [K(\frac{x-X_i}{h})]$. If I define $v = X_i$ and $u = \frac{x-v}{h}$ and substitute in that order, then I get $= \frac{1}{h} \int K(u) f(v)dv$. We know from (1) that $K(u)$ converges to $1$, so then we end up with $= \frac{1}{h} \int f(v)dv$. The only thing that still confuses me for (2) is how I'm supposed to get those limits? – inkalchemist1994 Mar 13 '19 at 20:54
• It doesn't make any sense to write "$K(u)$ converges to" anything: it's a function, pure and simple. You can graph it. – whuber Mar 13 '19 at 20:56

## 1 Answer

1. A bona fide kernel is a kernel that is non-negative and integrates to $$1$$. Since the values of $$K(u)$$ are explicitly restricted to $$\left\{0, 1\right\}$$, the boxcar kernel is non-negative. Now we determine if the boxcar kernel integrates to $$1$$:

\begin{equation*} \begin{aligned} \widehat{f}_n(x;h) &= \frac{1}{nh}\sum_{i=1}^n K\left(\frac{x-X_i}{h}\right) \\ \int_{-\infty}^{\infty} \widehat{f}_n(x;h)dx &= \int_{-\infty}^{\infty} \Bigg[ \frac{1}{nh}\sum_{i=1}^n K\left(\frac{x-X_i}{h}\right)dx \Bigg] \\ &= \frac{1}{n} \sum_{i=1}^n \int_{-\infty}^{\infty} \Bigg[ \frac{1}{h}K\left(\frac{x-X_i}{h}\right)dx \Bigg] \\ \text{Let } u = \left(\frac{x-X_i}{h}\right): & & \\ &= \int_{-\infty}^{\infty} \Bigg[ \frac{1}{h}K\left(u\right)du \Bigg] = 1 \hspace{1em} \checkmark \end{aligned} \end{equation*}

1. Using the definition $$\mathbb{E}(\hat{f}(x)) = \int{K_h (u) f(v)dv}$$:

\begin{equation*} \begin{aligned} \widehat{f}_n(x;h) &= \frac{1}{nh}\sum_{i=1}^n K\left(\frac{x-X_i}{h}\right) & \\ \mathbb{E}[\widehat{f}_n(x;h)] &= \frac{1}{nh} \sum_{i=1}^n \mathbb{E} \Bigg[ K\left(\frac{x-X_i}{h}\right) \Bigg] \\ &= \frac{1}{h} \mathbb{E} \Bigg[ K\left(\frac{x-X_i}{h}\right) \Bigg] \\ \text{Let } u = \left(\frac{x-X_i}{h}\right): & \\ &= \frac{1}{h} \mathbb{E} \Bigg[K(u)\Bigg] \\ &= \frac{1}{h} \int \Bigg[K(u)f(X_i)dX_i\Bigg] \\ &= \frac{1}{h} \int_{x-(h/2)}^{x+(h/2)} \Bigg[f(X_i)dX_i\Bigg] \\ \text{Let } v = X_i: & \\ &= \frac{1}{h} \int_{x-(h/2)}^{x+(h/2)} \Bigg[f(v)dv\Bigg] \hspace{1em} \checkmark \end{aligned} \end{equation*}

1. Using the definition, $$\mathbb{V}(\hat{f}(x)) = \frac{1}{n} \Bigg[ \frac{1}{h} \mathbb{E}(\hat{f}(x)) - \Bigg( \mathbb{E}(\hat{f}(x)) \Bigg)^2 \Bigg]$$:

\begin{equation*} \begin{aligned} \widehat{f}_n(x;h) &= \frac{1}{nh}\sum_{i=1}^n K\left(\frac{x-X_i}{h}\right) \\ \mathbb{V}(\widehat{f}_n(x;h)) &= \frac{1}{n} \Bigg[ \frac{1}{h} \mathbb{E}(\widehat{f}_n(x;h)) - \Bigg( \mathbb{E}(\widehat{f}_n(x;h)) \Bigg)^2 \Bigg] \\ \text{From previous question:} & \\ &= \frac{1}{n} \Bigg[ \frac{1}{h} \frac{1}{h} \int_{x-(h/2)}^{x+(h/2)}{f(v)dv} - \Bigg( \frac{1}{h} \int_{x-(h/2)}^{x+(h/2)}{f(v)dv}\Bigg)^2 \Bigg] \\ &= \frac{1}{n} \Bigg[ \bigg(\frac{1}{h}\bigg)^2 \int_{x-(h/2)}^{x+(h/2)}{f(v)dv} - \bigg(\frac{1}{h}\bigg)^2 \Bigg( \int_{x-(h/2)}^{x+(h/2)}{f(v)dv}\Bigg)^2 \Bigg] \\ &= \frac{1}{nh^2} \Bigg[ \int_{x-(h/2)}^{x+(h/2)}{f(v)dv} - \Bigg( \int_{x-(h/2)}^{x+(h/2)}{f(v)dv}\Bigg)^2 \Bigg] \hspace{1em} \checkmark \end{aligned} \end{equation*}