2
$\begingroup$

I am studying about Kernels in Kernel Density Estimation and I came to understand that the bigger the $n$ that satisfies $\int_{-\infty}^{\infty}K(u)\cdot u^{j}du=0$ for all $1\leq j \leq n$ the more the bound on the bias is smaller in the MSE.

I have searched about Kernel with $n\geq 2$ and so far I only found found Silverman kernel: $K(u)=\frac{1}{2}e^{\frac{-|u|}{2}}\cdot sin(\frac{|u|}{2}+\frac{\pi}{4})$ in here Kernel which have a Variance equal to zero apparently ($\int_{-\infty}^{\infty}K(u) \cdot u^{2}du=0$).

My question is divided in two parts:

  • how to prove that Silverman kernel has a zero variance
  • what are the other kernels with zero variance or generally with $n\geq 2$
$\endgroup$
3
  • $\begingroup$ There is an infinite Hilbert space of answers to your second question. Also, have you noticed that any such kernel must have some negative values and thereby does not satisfy your reference's very definition of a kernel? $\endgroup$
    – whuber
    May 5, 2021 at 21:01
  • $\begingroup$ @whuber, it does seem like an infinite Hilbert space, but I had some hard time imagining what they could be, especially that the Silverman kernel seemed so complicated As for the for my reference I am using Introduction to nonparametric estimation, exactly Proposition 1.2 $\endgroup$
    – Lazag
    May 6, 2021 at 3:18
  • $\begingroup$ $b(x_{0})=\int K(u) \frac{(u h)^{n}}{n !}\left(p^{(n)}\left(x_{0}+\tau u h\right)-p^{(n)}\left(x_{0}\right)\right) du $ @whuber from this I thought that the bigger the n the better we have approximation of $p(x_{0})$ if p is originally smooth $\endgroup$
    – Lazag
    May 6, 2021 at 3:28

2 Answers 2

2
$\begingroup$

Since the expressions for $K$ in your question and your answer are a little different, let's address a slight generalization. You are concerned with a positive number $\lambda=1/\sqrt{2}$ and a real "phase" $\phi$ for which the Kernel is (some multiple of)

$$K(u,\lambda,\phi) = \frac{1}{2}\exp\left(-\lambda |u|\right)\,\sin\left(\lambda |u|+\phi\right).$$

For any power $k\ge 0$ you wish to compute

$$\mu_k(\lambda,\phi) = \int_{\mathbb{R}}u^k\,K(u;\lambda,\phi)\,\mathrm{d}u.$$

(Graphs of the integrands for $k=0,1,2,3$ appear at the end of this post, shaded to show the areas in question and scaled to make them comparable.)

The simplest way perhaps is to compute the characteristic function of $K.$ However, there is a fairly elementary derivation that is more direct. It requires only two familiar definitions, a basic property of $\exp,$ and a simple substitution in the integral.

First note that when $k$ is an odd integer, the integrand is an odd function of $u$ (and obviously converges), whence it evaluates to zero.

When $k$ is an even integer, the integrand is an even function of $u,$ so we can just evaluate it from $0$ to $\infty$ and double that.

Employing the definition $\sin(z) = \Im\left[\exp(iz)\right]$ (the imaginary part of the Complex exponential) and the integral expression for the Gamma function

$$\Gamma(z) = \int_0^\infty x^z\,e^{-x}\,\frac{\mathrm{d}x}{x},$$

perform the computation by substituting $x = \lambda(1-iu):$

$$\begin{aligned} \mu_k(\lambda,\phi) &= 2\int_0^\infty u^k\,\frac{1}{2}\exp\left(-\lambda |u|\right)\,\sin\left(\lambda |u|+\phi\right)\,\mathrm{d}u\ \\ &= \int_0^\infty u^k\,\exp\left(-\lambda u\right)\,\Im\left[ \exp\left(i[\lambda u+\phi\right])\right]\,\mathrm{d}u\\ &= \Im\left[ e^{i\phi}\int_0^\infty u^{k+1}\,\exp\left(-\lambda(1+i)u\right)\right]\,\frac{\mathrm{d}u}{u}\\ &= \Im\left[e^{i\phi}\,\left(\lambda(1+i)\right)^{-(k+1)}\,\Gamma(k+1)\right]\\ &= \lambda^{-(k+1)}\, \Gamma(k+1)\, \Im\left[e^{i\phi}(1+i)^{-(k+1)}\right]. \end{aligned}$$

A simplification (removing all references to the Complex unit $i$) is available because $1+i = \sqrt{2}\,e^{i\pi/4},$ giving

$$\mu_k(\lambda,\phi) = \lambda^{-(k+1)}\, 2^{-(k+1)/2}\, \Gamma(k+1)\, \sin\left(\frac{(k+1)\pi}{4} + \phi\right).$$

When (for instance) $\lambda = 1/\sqrt{2},$ the first two terms cancel; and when $\phi=\pi/4,$ the whole expression simplifies (via the relation $\sin(z + \pi/2) = \cos(z)$) to

$$\mu_k\left(1/\sqrt{2}, \pi/4\right) = \Gamma(k+1)\cos\left(\frac{\pi k}{4}\right);\quad k\text{ even}.$$

For even integral values of $k=0,2,4,6,\ldots,$ $\cos(\pi k/4)$ cycles through the numbers

$$1,\ 0,\ -1,\ 0,\ 1,\ \ldots$$

and then cyclically repeats.


The appearance of $\Gamma(k+1) = k!$ in the answer suggests visualizing $\mu_k/k!$ instead of $\mu_k$ itself. Here are graphs of $u^k\,K(u;1/\sqrt{2},\pi/4)/k!$ for $k=0,1,2,3.$ The kernel $K$ itself corresponds to $k=0$ and appears in blue, spiking at the origin. As $k$ increases, the graphs swing more wildly; the red curve is for $k=3.$

Figure

$\endgroup$
1
  • $\begingroup$ thanks for the very detailed explanation, I guess I forgot the root when asking the question, but I guess I should leave like this for other people to learn from your answer ! so in your in answer for k=4 we don't get zero right ? meaning the oder of this kernel is 3 ? $\endgroup$
    – Lazag
    May 8, 2021 at 13:36
0
$\begingroup$

I have found an answer for my first question:

Silverman kernel is of order $j=3$(at least, haven't checked for $j > 3$)

$K(x)=\frac{1}{2}e^{\left(\frac{-|x|}{\sqrt{2}}\right)}sin\left(\frac{|x|}{\sqrt{2}}+\frac{\pi}{4}\right)$

    1. $\int_{-\infty}^{\infty}K(x)dx=1$
    1. $\int_{-\infty}^{\infty}xK(x)dx=0$
    1. $\int_{-\infty}^{\infty}x^{2}K(x)dx=0$
    1. $\int_{-\infty}^{\infty}x^{3}K(x)dx=0$

$\text{1.Proof}$:

\begin{aligned} \int_{-\infty}^{\infty}K(x)dx&=\int_{-\infty}^{\infty}\frac{1}{2}e^{\left(\frac{-|x|}{\sqrt{2}}\right)}sin\left(\frac{|x|}{\sqrt{2}}+\frac{\pi}{4}\right)dx\\ &=\int_{0}^{\infty}e^{\left(\frac{-x}{\sqrt{2}}\right)}sin\left(\frac{x}{\sqrt{2}}+\frac{\pi}{4}\right)dx\\ &=\sqrt{2}\int_{0}^{\infty}e^{\left(-x\right)}sin\left(-x+\frac{\pi}{4}\right)dx\\ \end{aligned}

$$ =\sqrt{2}\left(-\mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right)\Biggr|_{0}^{\infty}-\left(\mathrm{e}^{-x} \cos \left(x+\frac{\pi}{4}\right)\Biggr|_{0}^{\infty}- \int_{0}^{\infty}-\mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right) \mathrm{d} x\right)\right) $$ $$ =\sqrt{2}\left(-\mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right)\Biggr|_{0}^{\infty}-\left(\mathrm{e}^{-x} \cos \left(x+\frac{\pi}{4}\right)\Biggr|_{0}^{\infty}+\int_{0}^{\infty} \mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right) \mathrm{d} x\right)\right) $$

$$ =\frac{-\mathrm{e}^{-x} \sin \left(x+\frac{\pi}{4}\right)-\mathrm{e}^{-x} \cos \left(x+\frac{\pi}{4}\right)}{\sqrt{2}}\Biggr|_{0}^{\infty}=1 $$ 2.$\text{Proof}$: $$ -xK(-x)=-xK(x)\Longrightarrow \int_{-\infty}^{\infty}xK(x)dx=0 $$ 3.$\text{Proof}$: $$ \begin{aligned} \int_{-\infty}^{\infty}x^{2}K(x)dx&=\int_{-\infty}^{\infty}\frac{1}{2}x^{2}e^{\left(\frac{-|x|}{\sqrt{2}}\right)}sin\left(\frac{|x|}{\sqrt{2}}+\frac{\pi}{4}\right)dx\\ &=\int_{0}^{\infty}x^{2}e^{\left(\frac{-x}{\sqrt{2}}\right)}sin\left(\frac{x}{\sqrt{2}}+\frac{\pi}{4}\right)dx\\ &=2\sqrt{2}\int_{0}^{\infty}x^{2}e^{\left(-x\right)}sin\left(x+\frac{\pi}{4}\right)dx\\ &=2\sqrt{2}\dfrac{\left(x+1\right)\mathrm{e}^{-x}\left(\sin\left(x\right)-x\cos\left(x\right)\right)}{\sqrt{2}}\Biggr|_{0}^{\infty}=0 \end{aligned} $$ 4.Proof: Similar to the proof of 2

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.