what are the Kernels with zero variance (in KDE)?

Question

I am studying about Kernels in Kernel Density Estimation and I came to understand that the bigger the $n$ that satisfies $\int_{-\infty}^{\infty}K(u)\cdot u^{j}du=0$ for all $1\leq j \leq n$ the more the bound on the bias is smaller in the MSE.

I have searched about Kernel with $n\geq 2$ and so far I only found found Silverman kernel: $K(u)=\frac{1}{2}e^{\frac{-|u|}{2}}\cdot sin(\frac{|u|}{2}+\frac{\pi}{4})$ in here Kernel which have a Variance equal to zero apparently ($\int_{-\infty}^{\infty}K(u) \cdot u^{2}du=0$).

My question is divided in two parts:

• how to prove that Silverman kernel has a zero variance
• what are the other kernels with zero variance or generally with $n\geq 2$

Answer 1

Since the expressions for $K$ in your question and your answer are a little different, let's address a slight generalization. You are concerned with a positive number $\lambda=1/\sqrt{2}$ and a real "phase" $\phi$ for which the Kernel is (some multiple of)

$$K(u,\lambda,\phi) = \frac{1}{2}\exp\left(-\lambda |u|\right)\,\sin\left(\lambda |u|+\phi\right).$$

For any power $k\ge 0$ you wish to compute

$$\mu_k(\lambda,\phi) = \int_{\mathbb{R}}u^k\,K(u;\lambda,\phi)\,\mathrm{d}u.$$

(Graphs of the integrands for $k=0,1,2,3$ appear at the end of this post, shaded to show the areas in question and scaled to make them comparable.)

The simplest way perhaps is to compute the characteristic function of $K.$ However, there is a fairly elementary derivation that is more direct. It requires only two familiar definitions, a basic property of $\exp,$ and a simple substitution in the integral.

First note that when $k$ is an odd integer, the integrand is an odd function of $u$ (and obviously converges), whence it evaluates to zero.

When $k$ is an even integer, the integrand is an even function of $u,$ so we can just evaluate it from $0$ to $\infty$ and double that.

Employing the definition $\sin(z) = \Im\left[\exp(iz)\right]$ (the imaginary part of the Complex exponential) and the integral expression for the Gamma function

$$\Gamma(z) = \int_0^\infty x^z\,e^{-x}\,\frac{\mathrm{d}x}{x},$$

perform the computation by substituting $x = \lambda(1-iu):$

$$\begin{aligned} \mu_k(\lambda,\phi) &= 2\int_0^\infty u^k\,\frac{1}{2}\exp\left(-\lambda |u|\right)\,\sin\left(\lambda |u|+\phi\right)\,\mathrm{d}u\ \\ &= \int_0^\infty u^k\,\exp\left(-\lambda u\right)\,\Im\left[ \exp\left(i[\lambda u+\phi\right])\right]\,\mathrm{d}u\\ &= \Im\left[ e^{i\phi}\int_0^\infty u^{k+1}\,\exp\left(-\lambda(1+i)u\right)\right]\,\frac{\mathrm{d}u}{u}\\ &= \Im\left[e^{i\phi}\,\left(\lambda(1+i)\right)^{-(k+1)}\,\Gamma(k+1)\right]\\ &= \lambda^{-(k+1)}\, \Gamma(k+1)\, \Im\left[e^{i\phi}(1+i)^{-(k+1)}\right]. \end{aligned}$$

A simplification (removing all references to the Complex unit $i$) is available because $1+i = \sqrt{2}\,e^{i\pi/4},$ giving

$$\mu_k(\lambda,\phi) = \lambda^{-(k+1)}\, 2^{-(k+1)/2}\, \Gamma(k+1)\, \sin\left(\frac{(k+1)\pi}{4} + \phi\right).$$

When (for instance) $\lambda = 1/\sqrt{2},$ the first two terms cancel; and when $\phi=\pi/4,$ the whole expression simplifies (via the relation $\sin(z + \pi/2) = \cos(z)$) to

$$\mu_k\left(1/\sqrt{2}, \pi/4\right) = \Gamma(k+1)\cos\left(\frac{\pi k}{4}\right);\quad k\text{ even}.$$

For even integral values of $k=0,2,4,6,\ldots,$ $\cos(\pi k/4)$ cycles through the numbers

$$1,\ 0,\ -1,\ 0,\ 1,\ \ldots$$

and then cyclically repeats.

---

The appearance of $\Gamma(k+1) = k!$ in the answer suggests visualizing $\mu_k/k!$ instead of $\mu_k$ itself. Here are graphs of $u^k\,K(u;1/\sqrt{2},\pi/4)/k!$ for $k=0,1,2,3.$ The kernel $K$ itself corresponds to $k=0$ and appears in blue, spiking at the origin. As $k$ increases, the graphs swing more wildly; the red curve is for $k=3.$

Answer 2

I have found an answer for my first question:

Silverman kernel is of order $j=3$(at least, haven't checked for $j > 3$)

$K(x)=\frac{1}{2}e^{\left(\frac{-|x|}{\sqrt{2}}\right)}sin\left(\frac{|x|}{\sqrt{2}}+\frac{\pi}{4}\right)$

• 1. $\int_{-\infty}^{\infty}K(x)dx=1$
• 2. $\int_{-\infty}^{\infty}xK(x)dx=0$
• 3. $\int_{-\infty}^{\infty}x^{2}K(x)dx=0$
• 4. $\int_{-\infty}^{\infty}x^{3}K(x)dx=0$

$\text{1.Proof}$:

\begin{aligned} \int_{-\infty}^{\infty}K(x)dx&=\int_{-\infty}^{\infty}\frac{1}{2}e^{\left(\frac{-|x|}{\sqrt{2}}\right)}sin\left(\frac{|x|}{\sqrt{2}}+\frac{\pi}{4}\right)dx\\ &=\int_{0}^{\infty}e^{\left(\frac{-x}{\sqrt{2}}\right)}sin\left(\frac{x}{\sqrt{2}}+\frac{\pi}{4}\right)dx\\ &=\sqrt{2}\int_{0}^{\infty}e^{\left(-x\right)}sin\left(-x+\frac{\pi}{4}\right)dx\\ \end{aligned}

$$ =\sqrt{2}\left(-\mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right)\Biggr|_{0}^{\infty}-\left(\mathrm{e}^{-x} \cos \left(x+\frac{\pi}{4}\right)\Biggr|_{0}^{\infty}- \int_{0}^{\infty}-\mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right) \mathrm{d} x\right)\right) $$ $$ =\sqrt{2}\left(-\mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right)\Biggr|_{0}^{\infty}-\left(\mathrm{e}^{-x} \cos \left(x+\frac{\pi}{4}\right)\Biggr|_{0}^{\infty}+\int_{0}^{\infty} \mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right) \mathrm{d} x\right)\right) $$

$$ =\frac{-\mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right)-\mathrm{e}^{-x} \cos \left(x+\frac{\pi}{4}\right)}{\sqrt{2}}\Biggr|_{0}^{\infty}=1 $$ 2.$\text{Proof}$: $$ -xK(-x)=-xK(x)\Longrightarrow \int_{-\infty}^{\infty}xK(x)dx=0 $$ 3.$\text{Proof}$: $$ \begin{aligned} \int_{-\infty}^{\infty}x^{2}K(x)dx&=\int_{-\infty}^{\infty}\frac{1}{2}x^{2}e^{\left(\frac{-|x|}{\sqrt{2}}\right)}sin\left(\frac{|x|}{\sqrt{2}}+\frac{\pi}{4}\right)dx\\ &=\int_{0}^{\infty}x^{2}e^{\left(\frac{-x}{\sqrt{2}}\right)}sin\left(\frac{x}{\sqrt{2}}+\frac{\pi}{4}\right)dx\\ &=2\sqrt{2}\int_{0}^{\infty}x^{2}e^{\left(-x\right)}sin\left(x+\frac{\pi}{4}\right)dx\\ &=2\sqrt{2}\dfrac{\left(x+1\right)\mathrm{e}^{-x}\left(\sin\left(x\right)-x\cos\left(x\right)\right)}{\sqrt{2}}\Biggr|_{0}^{\infty}=0 \end{aligned} $$ 4.Proof: Similar to the proof of 2