# Moments of kernel density estimates

Assume that I have a random sample $X_1,\dots,X_m$ from some distribution $F$, and I want to estimate the first two moments of $F$. Obviously, any sane person would use the sample moments, but I was wondering about the following situation:

Instead of using the sample moments, I employ kernel density estimation, for example with a Gaussian kernel, which leaves me with some distribution, in this case a mixutre of normal distributions due to teh Gaussian kernel. Now, my question is: Are there any results on conditions under which the mean and variance of the "KDE-distribution" converge to the true distribution $F$? How does this depend on kernel, bandwidth, or properties of the sample?

I wasn't able to find research on this as usually, questions such as "What is the expectation of the KD-estimate at a point $x$?" seem to be addressed, rather than the moments of the resulting distribution.

• KDE introduces external noise into your data so it will have variance greater then in your initial sample. However you can force it to match the first two moments. – Tim Mar 16 '17 at 11:16
• @Tim How can a KD-estimate be forced to match the first two moments? -- and isn't the variance fully determined by the first two moments? – user3825755 Mar 16 '17 at 12:48

Moments are reasonably straightforward via the law of total expectation and the law of total variance:

1. Law of total expectation

$$\operatorname {E} (Y)=\operatorname {E} (\operatorname {E} (Y\mid X)),$$

Here the $X$ is the original variable and $Y|X$ can be thought of as the kernel. The unconditional distribution is the resulting KDE.

We have $E(Y|X) = X$, and so $E(Y) = E(X) = \mu_F$.

2. Law of total variance

$$\operatorname {Var} (Y)=\operatorname {E} [\operatorname {Var} (Y|X)]+\operatorname {Var} (\operatorname {E} [Y|X]).$$

$\text{Var}(Y|X)$ is the variance of the kernel at $X$, which is constant, so $E(Var(Y|X))=\sigma^2_K = h^2$.

The second term reduces to $\text{Var}(X)=\sigma^2_F$

So $\text{Var}(Y) = \sigma^2_F + h^2$.

In summary, the expectation is unaffected by a zero-mean kernel, but the variance is (unsurprisingly) increased by the variance of the kernel.

• Actually, let me add another comment if you allow: All these considerations are independent of the actual properties of the sample, right? So if the sample is not independent, then all this should still hold, right? – user3825755 Mar 16 '17 at 13:19