How Parzen window density estimate $f_n$ converges to f

Question

I am trying to understand how Parzen window density estimate converges to actual density function f(x).[Actually i am trying to learn machine learning on my own using available free resources. Please help me in the below]

Let $f_n(x)$ be the Parzen window density estimate of actual density f(x). Given $x_1,x_2....x_n $ are iid sample (given training data).

Let h be parameter. $V_n$ be volume (say hypercube). Now in Parzen, we take estimate for density function to be linear sum of kernel functions at sample points. To show that estimate converges to actual f(x), i did in following way(for each sample size n, $V_n, h_n$ , varies and also as $n \to \infty, h_n \to 0, V_n \to 0, $ but $n V_n \to 0$)

$E(\hat f_n(x)) = \frac {1}{n} \sum_{i=1}^n E(\frac{1}{V_n} \phi(\frac{x-x_i}{h_n})) \\ = E(\frac{1}{V_n} \phi(\frac{x-x_i}{h_n})) \\= \int \frac{1}{V_n} \phi(\frac{x-x_i}{h_n}) f(z) dz$ (as each term expectation is same and $\phi $ be some kernel function, f be density )

Above Last integral be (1)

After that how to proceed? I am following https://www.youtube.com/watch?v=esoVuEG-X1I&list=PLbMVogVj5nJSlpmy0ni_5-RgbseafOViy&index=13&t=2617s (At 26.01 )

Here sir says this integral (1) goes to f(x) as $n \to \infty$ but i did not understand how.

I know

$\int \frac{1}{V_n}\phi(\frac{x-x_i}{h_n}) dz = 1$ (since $\phi$ is kernel function)

Also, i tried to expand on final integral(1) using integration by parts

then $f(z)\int \frac{1}{V_n}\phi(\frac{x-x_i}{h_n}) dz - \int f'(z) \int \frac{1}{V_n}\phi(\frac{x-x_i}{h_n}) dz dz = f(z) - f(z)=0$ (as integral of kernel function sums to 1)

Please explain where i did it wrongly or understood wrongly.

Answer 1

First, they define the following function: $$\delta_n(x)=\frac{1}{V_n}\phi\left({x\over h_n}\right)$$ and this is assumed (check 22m39s) to converge to delta function as $n\rightarrow\infty$. So, basically any kernel that doesn't satisfy this convergence property, even if it converges to the true density, we can't prove it this way. That said,

$$E[\hat f_n (x)]=\int_{-\infty}^\infty \delta_n(x-z)f(z)dz\rightarrow \int_{-\infty}^\infty \delta(x-z)f(z)dz=\int_{x^-}^{x^+}\delta(x-z)f(z)dz=f(x)$$

The last two follows from the properties of delta function.