# How Parzen window density estimate $f_n$ converges to f

I am trying to understand how Parzen window density estimate converges to actual density function f(x).[Actually i am trying to learn machine learning on my own using available free resources. Please help me in the below]

Let $$f_n(x)$$ be the Parzen window density estimate of actual density f(x). Given $$x_1,x_2....x_n$$ are iid sample (given training data).

Let h be parameter. $$V_n$$ be volume (say hypercube). Now in Parzen, we take estimate for density function to be linear sum of kernel functions at sample points. To show that estimate converges to actual f(x), i did in following way(for each sample size n, $$V_n, h_n$$ , varies and also as $$n \to \infty, h_n \to 0, V_n \to 0,$$ but $$n V_n \to 0$$)

$$E(\hat f_n(x)) = \frac {1}{n} \sum_{i=1}^n E(\frac{1}{V_n} \phi(\frac{x-x_i}{h_n})) \\ = E(\frac{1}{V_n} \phi(\frac{x-x_i}{h_n})) \\= \int \frac{1}{V_n} \phi(\frac{x-x_i}{h_n}) f(z) dz$$ (as each term expectation is same and $$\phi$$ be some kernel function, f be density )

Above Last integral be (1)

After that how to proceed? I am following https://www.youtube.com/watch?v=esoVuEG-X1I&list=PLbMVogVj5nJSlpmy0ni_5-RgbseafOViy&index=13&t=2617s (At 26.01 )

Here sir says this integral (1) goes to f(x) as $$n \to \infty$$ but i did not understand how.

I know

$$\int \frac{1}{V_n}\phi(\frac{x-x_i}{h_n}) dz = 1$$ (since $$\phi$$ is kernel function)

Also, i tried to expand on final integral(1) using integration by parts

then $$f(z)\int \frac{1}{V_n}\phi(\frac{x-x_i}{h_n}) dz - \int f'(z) \int \frac{1}{V_n}\phi(\frac{x-x_i}{h_n}) dz dz = f(z) - f(z)=0$$ (as integral of kernel function sums to 1)

Please explain where i did it wrongly or understood wrongly.

First, they define the following function: $$\delta_n(x)=\frac{1}{V_n}\phi\left({x\over h_n}\right)$$ and this is assumed (check 22m39s) to converge to delta function as $$n\rightarrow\infty$$. So, basically any kernel that doesn't satisfy this convergence property, even if it converges to the true density, we can't prove it this way. That said,
$$E[\hat f_n (x)]=\int_{-\infty}^\infty \delta_n(x-z)f(z)dz\rightarrow \int_{-\infty}^\infty \delta(x-z)f(z)dz=\int_{x^-}^{x^+}\delta(x-z)f(z)dz=f(x)$$
• Your expansion seems quite complicated, because you don't use limits, and confuse $x_i$ and $z$'s. I feel that first integral will be $0$. – gunes Mar 19 at 14:35