We often try to minimize the MISE of a KDE: $\text{E}_{\mathbb{P}^n}[\int (\hat{p}(x) - p(x))^2 dx]$. Why don't we instead try to minimize $\text{E}_{\mathbb{P}^n}[\int (\hat{p}(x) - p(x))^2 p(x) dx]$, so that we weight deviations from the true density more when they occur in high-probability regions?


1 Answer 1


The MISE is designed to give a measure of error of an estimator $\hat{p}$ from a true density $p$. For a fixed estimate, that distance can be obtained as the squared-error, which is the internal integral in the expectation. In the standard MISE we do not weight this integral, since we consider errors to be of the same importance regardless of where they occur in the density.

Since the MISE is an expectation, it is implicitly an integral over the estimator $\hat{p}$. If we explicitly show that $\hat{p}$ depends on the data vector $\mathbf{x}$ (which is different to the argument $x$) then we have:

$$\begin{equation} \begin{aligned} \text{MISE}(\hat{p}) &\equiv \mathbb{E} \Big[ \int (\hat{p}(x,\mathbf{X}) - p(x))^2 dx \Big] \\[6pt] &= \int \Big( \int (\hat{p}(x,\mathbf{x}) - p(x))^2 dx \Big) f(\mathbf{x}) d\mathbf{x} \\[6pt] &= \int \int (\hat{p}(x,\mathbf{x}) - p(x))^2 f(\mathbf{x}) dx d\mathbf{x}. \\[6pt] \end{aligned} \end{equation}$$

So as you can see, the MISE does involve an integral of the weighted squared-error, but the weighting is with respect to the density for the data that was used in the estimator $\hat{p}$.

It is of course possible to modify the MISE to weight the squared-error integral, to make some parts of the density count as being more important than other parts. This would be appropriate if you want to estimate more accurately in some parts of the support than others. In such a case you could apply a weighting to get a weighted mean integrated squared-error (WMISE).

  • $\begingroup$ I suppose that in expectation the estimate will be closer to the truth in regions of high density than low density anyway, right? So I can't tell if weighting would really be of interest... $\endgroup$ Commented Jul 9, 2018 at 17:03
  • 1
    $\begingroup$ Not really, since the expectation is a weighting over the variable $\mathbf{x}$, not $x$. If you would like to estimate better in particular regions (and worse) in others, you could apply a separate weighting function on $x$ as you suggested in your post. It is really a matter of how you want to measure the distance between two distributions $\hat{p}$ and $p$. $\endgroup$
    – Ben
    Commented Jul 9, 2018 at 23:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.