# Consistency of density estimation under marginalization

Let $(x_1,y_1),\ldots,(x_n,y_n)$ be samples from some unknown distribution $p(X,Y)$ and $\hat{p}(X,Y)$, $\hat{p}(Y)$ density estimates of the joint and marginal distributions (i.e., for the estimation of $\hat{p}(Y)$ we only use the $y$-part of the samples).

Question: Under which condition is

$\int_x \hat{p}(x,y)\ \mathrm{d}x = \hat{p}(y)$?

The answer obviously depends on the estimator. Is it true only in the limit and for consistent estimators? Is it important whether the estimate is parametric or non-parametric?

Essentially, I want to understand why marginalization could be useful in a continuous and non-parametric setting, if I can just as well learn the marginal distribution from the restriction of the data to the marginal dimensions.

• without details about the estimation method you use, the question is unclear as we can always define $\hat{p}(y)$ by $$\int_\mathcal{X} \hat{p}(x,y)\ \mathrm{d}x = \hat{p}(y)$$ – Xi'an Feb 4 '15 at 6:58
• No, you can't. As I wrote, $\hat{p}(y)$ can only use the $y$-part of the samples. I can't be more specific, because the conditions are part of the question. You can assume kernel density estimation, if you need to. – ASML Feb 4 '15 at 7:04

Suppose further that $k_{xy}\left( \begin{bmatrix}x \\ y\end{bmatrix}, \begin{bmatrix}x_i \\ y_i\end{bmatrix} \right) = k_y(y, y_i) \, k_{x \mid y}(x, x_i \mid y, y_i)$ for some function $k_{x \mid y}$ which integrates to 1 for any $y, y_i$. If the kernel function is a probability density (i.e. is everywhere nonnegative), then this should hold, since $k_{x \mid y}$ is itself a marginal density; even if not, a product kernel will still satisfy this.