3
$\begingroup$

I'm studying about kernel density estimation and from wikipedia I get this formula:

$$\hat{f}_h(x, h) = \frac{1}{n}\sum^{n}_{i=1}K_h(x-x_i) = \frac{1}{nh}\sum_{i=1}^nK(\frac{x-x_i}{h}).$$

I think I've got the basic idea of this formula, but the smoothing term $h$ is what troubles me. If my kernel function was the Gaussian:

$$K(u)=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}u^2}$$

I would get my estimator to be:

$$\hat{f}_h(x, h) = \frac{1}{nh\sqrt{2\pi}}\sum_{i=1}^n e^{-\frac{1}{2}u^2},$$

where $\displaystyle u = \frac{x-x_i}{h}.$ Why is the $h$ term in this formula? What is its function? Why do we divide the sum by $h$ etc. Can someone clarify this a bit?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Notice that your density function is an average of density functions of gaussians with standard deviation $h$ and different centers. Does that clarify why you're using this formula? $\endgroup$
    – Jundiaius
    Commented May 7, 2014 at 7:45
  • $\begingroup$ +1 @eqperes yes I thought so that $h$ is related to the standard deviation. So does the sum $\sum e^{(-1/2) u^2} = nh\sqrt{2\pi}$ in the maximum case? Sry if my question is dumb, I'm bit mixed up with this. $\endgroup$
    – jjepsuomi
    Commented May 7, 2014 at 7:52
  • 1
    $\begingroup$ Well, no. That will depend on all your $x_{i}$ points. But, for me to understand better your questions, why do you think it should be equal to $nh\sqrt{2\pi}$ in the maximum? $\endgroup$
    – Jundiaius
    Commented May 7, 2014 at 8:13
  • $\begingroup$ +1 darmet x) sry @eqperes I thought about the function as a probability all the time and not as a probability density. That's why I thought the sum should equal $nh\sqrt{2\pi}$ in the maximun case so that the overall value of the function would (when divided with the normalizing term $1/nh\sqrt{2\pi}$) equal probability value 1. I need to further examine this function, I think I don't understand it yet well enough to make clear questions about it. Maybe you could explain about this function as an answer? I could then accept it as an answer? Thank you for your effort! :) $\endgroup$
    – jjepsuomi
    Commented May 7, 2014 at 8:36
  • $\begingroup$ +1 @eqperes I got it =) no need to answer anymore, I integrated the density and it clarified it :) $\endgroup$
    – jjepsuomi
    Commented May 7, 2014 at 9:10

1 Answer 1

Reset to default
3
$\begingroup$

I got the answer myself already, no need to answer :) Integrating the density function answered to all my questions :)

$$K(y) = \frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}y^2}$$

If we have the $n$ observations: $\mu_1, ..., \mu_n$ and $y_i = \displaystyle \frac{x-\mu_i}{h}$, then

$$\int_{-\infty}^{\infty}\hat{f}_h(x)\;dx=\int_{-\infty}^{\infty}\frac{1}{nh}\sum_{i=1}^nK(y_i)\;dx = \int_{-\infty}^{\infty}\frac{1}{nh}\sum_{i=1}^nK(\frac{x-\mu_i}{h})\;dx$$

$$=\frac{1}{nh\sqrt{2\pi}}\int_{-\infty}^{\infty}\sum_{i=1}^n e^{-\frac{1}{2}(\frac{x-\mu_i}{h})^2}\;dx = \frac{1}{nh\sqrt{2\pi}}\left[ \int_{-\infty}^{\infty}e^{-\frac{1}{2}(\frac{x-\mu_1}{h})^2}+\cdots+\int_{-\infty}^{\infty}e^{-\frac{1}{2}(\frac{x-\mu_n}{h})^2}\right] = \frac{1}{nh\sqrt{2\pi}}\cdot nh\sqrt{2\pi}=1.$$

Improve this answer
$\endgroup$
2
  • $\begingroup$ Can I ask you what the value X is representing in that formula? $\endgroup$
    – Catia Cannata
    Commented Sep 18, 2015 at 13:04
  • $\begingroup$ Hi @CatiaCannata, sry for my late reply. The $x$ represents the random variable with unknown density $f$ we want to estimate. Take a look at the wikipedia page I have in my question :) $\endgroup$
    – jjepsuomi
    Commented May 2, 2016 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.