6
$\begingroup$

Assume that I have a random sample $X_1,\dots,X_m$ from some distribution $F$, and I want to estimate the first two moments of $F$. Obviously, any sane person would use the sample moments, but I was wondering about the following situation:

Instead of using the sample moments, I employ kernel density estimation, for example with a Gaussian kernel, which leaves me with some distribution, in this case a mixutre of normal distributions due to teh Gaussian kernel. Now, my question is: Are there any results on conditions under which the mean and variance of the "KDE-distribution" converge to the true distribution $F$? How does this depend on kernel, bandwidth, or properties of the sample?

I wasn't able to find research on this as usually, questions such as "What is the expectation of the KD-estimate at a point $x$?" seem to be addressed, rather than the moments of the resulting distribution.

$\endgroup$
  • $\begingroup$ KDE introduces external noise into your data so it will have variance greater then in your initial sample. However you can force it to match the first two moments. $\endgroup$ – Tim Mar 16 '17 at 11:16
  • $\begingroup$ @Tim How can a KD-estimate be forced to match the first two moments? -- and isn't the variance fully determined by the first two moments? $\endgroup$ – user3825755 Mar 16 '17 at 12:48
6
$\begingroup$

Moments are reasonably straightforward via the law of total expectation and the law of total variance:

  1. Law of total expectation

    $${\displaystyle \operatorname {E} (Y)=\operatorname {E} (\operatorname {E} (Y\mid X)),}$$

    Here the $X$ is the original variable and $Y|X$ can be thought of as the kernel. The unconditional distribution is the resulting KDE.

    We have $E(Y|X) = X$, and so $E(Y) = E(X) = \mu_F$.

  2. Law of total variance

    $${\displaystyle \operatorname {Var} (Y)=\operatorname {E} [\operatorname {Var} (Y|X)]+\operatorname {Var} (\operatorname {E} [Y|X]).}$$

    $\text{Var}(Y|X)$ is the variance of the kernel at $X$, which is constant, so $E(Var(Y|X))=\sigma^2_K = h^2$.

    The second term reduces to $\text{Var}(X)=\sigma^2_F$

    So $\text{Var}(Y) = \sigma^2_F + h^2$.

In summary, the expectation is unaffected by a zero-mean kernel, but the variance is (unsurprisingly) increased by the variance of the kernel.

$\endgroup$
  • $\begingroup$ Actually, let me add another comment if you allow: All these considerations are independent of the actual properties of the sample, right? So if the sample is not independent, then all this should still hold, right? $\endgroup$ – user3825755 Mar 16 '17 at 13:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.