Properties of Kernel Density Estimators

Question

Given

Let $X \in \mathbb{R}$ be a real-valued random variable with theoretical probability density function (pdf) $f(x)$ and corresponding cumulative distribution function (cdf) $F(x)$. Let $X_1, X_2, \cdots, X_n$ a random sample of size $n$ drawn according to the distribution of $X$. Let $h>0$ be a positive real number and consider the kernel density estimator $\widehat{f}_n$ of $f$ given for every $x \in \mathcal{X}$ by \begin{eqnarray} \label{eq:kde:1d:1} \widehat{f}_n(x;h) = \frac{1}{nh}\sum_{i=1}^n{K\left(\frac{x-X_i}{h}\right)} \end{eqnarray} where the kernel in this case is the so-called boxcar kernel defined by \begin{eqnarray} \label{eq:kernel:boxcar} K(u) = \left\{\begin{array}{ll} 1 & -\frac{1}{2} < u < \frac{1}{2}\\ 0 & \texttt{Otherwise}. \end{array} \right. \end{eqnarray}

Questions

1. Prove that the boxcar kernel of $K(u)$ is a bona fide kernel.
2. Show that $$ \mathbb{E}(\widehat{f}_n(x; h)) = \frac{1}{h}\displaystyle \int_{x-(h/2)}^{x+(h/2)}{f(v)dv} $$
3. Show that $$ \mathbb{V}(\widehat{f}_n(x; h)) = \frac{1}{nh^2}\left[\int_{x-(h/2)}^{x+(h/2)}{f(v)dv}-\left(\int_{x-(h/2)}^{x+(h/2)}{f(v)dv}\right)^2\right] $$

Attempts

1. A bona fide kernel is one that is real, genuine, legitimate. i.e. a density estimation function that is non-negative and integrates to $1$. How do I show that a function is non-negative? I know that there is a relationship between the Bias of a density estimation function and its bona fide-ness, but I'm not sure what that relationship is. Any direction here would be helpful.

2. I know this by definition: $$\mathbb{E}(\hat{f}(x)) = \int \frac{1}{h} K \big( \frac{x-y}{h} \big) f(y)dy$$ So I can use that formula, but what is $h$, and how do I substitute $f(v)dv$ in place of $f(y)dy$?

3. Similarly, I know this by definition: $$\mathbb{V}(\hat{f}(x)) = \int \frac{1}{h^2} K \big( \frac{x-y}{h} \big)^2 f(y)dy - \Big( \frac{1}{h} \int K \big(\ \frac{x-y}{h} \big) f(y)dy \Big)^2$$ But I still have the same confusions: but what is $h$, and how do I substitute $f(v)dv$ in place of $f(y)dy$?

Thank you in advance for any help/clarification you can provide!

Answer 1

1. A bona fide kernel is a kernel that is non-negative and integrates to $1$. Since the values of $K(u)$ are explicitly restricted to $\left\{0, 1\right\}$, the boxcar kernel is non-negative. Now we determine if the boxcar kernel integrates to $1$:

\begin{equation*} \begin{aligned} \widehat{f}_n(x;h) &= \frac{1}{nh}\sum_{i=1}^n K\left(\frac{x-X_i}{h}\right) \\ \int_{-\infty}^{\infty} \widehat{f}_n(x;h)dx &= \int_{-\infty}^{\infty} \Bigg[ \frac{1}{nh}\sum_{i=1}^n K\left(\frac{x-X_i}{h}\right)dx \Bigg] \\ &= \frac{1}{n} \sum_{i=1}^n \int_{-\infty}^{\infty} \Bigg[ \frac{1}{h}K\left(\frac{x-X_i}{h}\right)dx \Bigg] \\ \text{Let } u = \left(\frac{x-X_i}{h}\right): & & \\ &= \int_{-\infty}^{\infty} \Bigg[ \frac{1}{h}K\left(u\right)du \Bigg] = 1 \hspace{1em} \checkmark \end{aligned} \end{equation*}

---

2. Using the definition $\mathbb{E}(\hat{f}(x)) = \int{K_h (u) f(v)dv}$:

\begin{equation*} \begin{aligned} \widehat{f}_n(x;h) &= \frac{1}{nh}\sum_{i=1}^n K\left(\frac{x-X_i}{h}\right) & \\ \mathbb{E}[\widehat{f}_n(x;h)] &= \frac{1}{nh} \sum_{i=1}^n \mathbb{E} \Bigg[ K\left(\frac{x-X_i}{h}\right) \Bigg] \\ &= \frac{1}{h} \mathbb{E} \Bigg[ K\left(\frac{x-X_i}{h}\right) \Bigg] \\ \text{Let } u = \left(\frac{x-X_i}{h}\right): & \\ &= \frac{1}{h} \mathbb{E} \Bigg[K(u)\Bigg] \\ &= \frac{1}{h} \int \Bigg[K(u)f(X_i)dX_i\Bigg] \\ &= \frac{1}{h} \int_{x-(h/2)}^{x+(h/2)} \Bigg[f(X_i)dX_i\Bigg] \\ \text{Let } v = X_i: & \\ &= \frac{1}{h} \int_{x-(h/2)}^{x+(h/2)} \Bigg[f(v)dv\Bigg] \hspace{1em} \checkmark \end{aligned} \end{equation*}

---

3. Using the definition, $\mathbb{V}(\hat{f}(x)) = \frac{1}{n} \Bigg[ \frac{1}{h} \mathbb{E}(\hat{f}(x)) - \Bigg( \mathbb{E}(\hat{f}(x)) \Bigg)^2 \Bigg]$:

\begin{equation*} \begin{aligned} \widehat{f}_n(x;h) &= \frac{1}{nh}\sum_{i=1}^n K\left(\frac{x-X_i}{h}\right) \\ \mathbb{V}(\widehat{f}_n(x;h)) &= \frac{1}{n} \Bigg[ \frac{1}{h} \mathbb{E}(\widehat{f}_n(x;h)) - \Bigg( \mathbb{E}(\widehat{f}_n(x;h)) \Bigg)^2 \Bigg] \\ \text{From previous question:} & \\ &= \frac{1}{n} \Bigg[ \frac{1}{h} \frac{1}{h} \int_{x-(h/2)}^{x+(h/2)}{f(v)dv} - \Bigg( \frac{1}{h} \int_{x-(h/2)}^{x+(h/2)}{f(v)dv}\Bigg)^2 \Bigg] \\ &= \frac{1}{n} \Bigg[ \bigg(\frac{1}{h}\bigg)^2 \int_{x-(h/2)}^{x+(h/2)}{f(v)dv} - \bigg(\frac{1}{h}\bigg)^2 \Bigg( \int_{x-(h/2)}^{x+(h/2)}{f(v)dv}\Bigg)^2 \Bigg] \\ &= \frac{1}{nh^2} \Bigg[ \int_{x-(h/2)}^{x+(h/2)}{f(v)dv} - \Bigg( \int_{x-(h/2)}^{x+(h/2)}{f(v)dv}\Bigg)^2 \Bigg] \hspace{1em} \checkmark \end{aligned} \end{equation*}