I'm studying about kernel density estimation and from wikipedia I get this formula:

$$\hat{f}_h(x, h) = \frac{1}{n}\sum^{n}_{i=1}K_h(x-x_i) = \frac{1}{nh}\sum_{i=1}^nK(\frac{x-x_i}{h}).$$

I think I've got the basic idea of this formula, but the smoothing term $h$ is what troubles me. If my kernel function was the Gaussian:


I would get my estimator to be:

$$\hat{f}_h(x, h) = \frac{1}{nh\sqrt{2\pi}}\sum_{i=1}^n e^{-\frac{1}{2}u^2},$$

where $\displaystyle u = \frac{x-x_i}{h}.$ Why is the $h$ term in this formula? What is its function? Why do we divide the sum by $h$ etc. Can someone clarify this a bit?

  • 1
    $\begingroup$ Notice that your density function is an average of density functions of gaussians with standard deviation $h$ and different centers. Does that clarify why you're using this formula? $\endgroup$ – Jundiaius May 7 '14 at 7:45
  • $\begingroup$ +1 @eqperes yes I thought so that $h$ is related to the standard deviation. So does the sum $\sum e^{(-1/2) u^2} = nh\sqrt{2\pi}$ in the maximum case? Sry if my question is dumb, I'm bit mixed up with this. $\endgroup$ – jjepsuomi May 7 '14 at 7:52
  • 1
    $\begingroup$ Well, no. That will depend on all your $x_{i}$ points. But, for me to understand better your questions, why do you think it should be equal to $nh\sqrt{2\pi}$ in the maximum? $\endgroup$ – Jundiaius May 7 '14 at 8:13
  • $\begingroup$ +1 darmet x) sry @eqperes I thought about the function as a probability all the time and not as a probability density. That's why I thought the sum should equal $nh\sqrt{2\pi}$ in the maximun case so that the overall value of the function would (when divided with the normalizing term $1/nh\sqrt{2\pi}$) equal probability value 1. I need to further examine this function, I think I don't understand it yet well enough to make clear questions about it. Maybe you could explain about this function as an answer? I could then accept it as an answer? Thank you for your effort! :) $\endgroup$ – jjepsuomi May 7 '14 at 8:36
  • $\begingroup$ +1 @eqperes I got it =) no need to answer anymore, I integrated the density and it clarified it :) $\endgroup$ – jjepsuomi May 7 '14 at 9:10

I got the answer myself already, no need to answer :) Integrating the density function answered to all my questions :)

$$K(y) = \frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}y^2}$$

If we have the $n$ observations: $\mu_1, ..., \mu_n$ and $y_i = \displaystyle \frac{x-\mu_i}{h}$, then

$$\int_{-\infty}^{\infty}\hat{f}_h(x)\;dx=\int_{-\infty}^{\infty}\frac{1}{nh}\sum_{i=1}^nK(y_i)\;dx = \int_{-\infty}^{\infty}\frac{1}{nh}\sum_{i=1}^nK(\frac{x-\mu_i}{h})\;dx$$

$$=\frac{1}{nh\sqrt{2\pi}}\int_{-\infty}^{\infty}\sum_{i=1}^n e^{-\frac{1}{2}(\frac{x-\mu_i}{h})^2}\;dx = \frac{1}{nh\sqrt{2\pi}}\left[ \int_{-\infty}^{\infty}e^{-\frac{1}{2}(\frac{x-\mu_1}{h})^2}+\cdots+\int_{-\infty}^{\infty}e^{-\frac{1}{2}(\frac{x-\mu_n}{h})^2}\right] = \frac{1}{nh\sqrt{2\pi}}\cdot nh\sqrt{2\pi}=1.$$

| cite | improve this answer | |
  • $\begingroup$ Can I ask you what the value X is representing in that formula? $\endgroup$ – Catia Cannata Sep 18 '15 at 13:04
  • $\begingroup$ Hi @CatiaCannata, sry for my late reply. The $x$ represents the random variable with unknown density $f$ we want to estimate. Take a look at the wikipedia page I have in my question :) $\endgroup$ – jjepsuomi May 2 '16 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.