Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm studying about kernel density estimation and from wikipedia I get this formula:

$$\hat{f}_h(x, h) = \frac{1}{n}\sum^{n}_{i=1}K_h(x-x_i) = \frac{1}{nh}\sum_{i=1}^nK(\frac{x-x_i}{h}).$$

I think I've got the basic idea of this formula, but the smoothing term $h$ is what troubles me. If my kernel function was the Gaussian:

$$K(u)=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}u^2}$$

I would get my estimator to be:

$$\hat{f}_h(x, h) = \frac{1}{nh\sqrt{2\pi}}\sum_{i=1}^n e^{-\frac{1}{2}u^2},$$

where $\displaystyle u = \frac{x-x_i}{h}.$ Why is the $h$ term in this formula? What is its function? Why do we divide the sum by $h$ etc. Can someone clarify this a bit?

share|improve this question
1  
Notice that your density function is an average of density functions of gaussians with standard deviation $h$ and different centers. Does that clarify why you're using this formula? –  eqperes May 7 at 7:45
    
+1 @eqperes yes I thought so that $h$ is related to the standard deviation. So does the sum $\sum e^{(-1/2) u^2} = nh\sqrt{2\pi}$ in the maximum case? Sry if my question is dumb, I'm bit mixed up with this. –  jjepsuomi May 7 at 7:52
1  
Well, no. That will depend on all your $x_{i}$ points. But, for me to understand better your questions, why do you think it should be equal to $nh\sqrt{2\pi}$ in the maximum? –  eqperes May 7 at 8:13
    
+1 darmet x) sry @eqperes I thought about the function as a probability all the time and not as a probability density. That's why I thought the sum should equal $nh\sqrt{2\pi}$ in the maximun case so that the overall value of the function would (when divided with the normalizing term $1/nh\sqrt{2\pi}$) equal probability value 1. I need to further examine this function, I think I don't understand it yet well enough to make clear questions about it. Maybe you could explain about this function as an answer? I could then accept it as an answer? Thank you for your effort! :) –  jjepsuomi May 7 at 8:36
    
+1 @eqperes I got it =) no need to answer anymore, I integrated the density and it clarified it :) –  jjepsuomi May 7 at 9:10

1 Answer 1

up vote 2 down vote accepted

I got the answer myself already, no need to answer :) Integrating the density function answered to all my questions :)

$$K(y) = \frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}y^2}$$

If we have the $n$ observations: $\mu_1, ..., \mu_n$ and $y_i = \displaystyle \frac{x-\mu_i}{h}$, then

$$\int_{-\infty}^{\infty}\hat{f}_h(x)\;dx=\int_{-\infty}^{\infty}\frac{1}{nh}\sum_{i=1}^nK(y_i)\;dx = \int_{-\infty}^{\infty}\frac{1}{nh}\sum_{i=1}^nK(\frac{x-\mu_i}{h})\;dx$$

$$=\frac{1}{nh\sqrt{2\pi}}\int_{-\infty}^{\infty}\sum_{i=1}^n e^{-\frac{1}{2}(\frac{x-\mu_i}{h})^2}\;dx = \frac{1}{nh\sqrt{2\pi}}\left[ \int_{-\infty}^{\infty}e^{-\frac{1}{2}(\frac{x-\mu_1}{h})^2}+\cdots+\int_{-\infty}^{\infty}e^{-\frac{1}{2}(\frac{x-\mu_n}{h})^2}\right] = \frac{1}{nh\sqrt{2\pi}}\cdot nh\sqrt{2\pi}=1.$$

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.