# Sampling from a marginal when full density is given

I want to sample from a distribution with density $$f(\mathbf x) = \int f(c) \prod_{i=1}^n f(x_i|c) dc$$ where $\mathbf x=(x_1,x_2,...,x_n)$. In my particular setup, is easy to sample according to the densities $f(c)$ and $f(x_i|c)$, but it is not obvious how to sample from the marginal given above.

Would the following work?

1. Sample $c$ according to $f(c)$.
2. Sample $x_i$ according to $f(x_i|c)$, for $i=1,2,...,n$.

Heuristic argument for why this should work:

• Obviously $f(\mathbf x,c) = f(\mathbf x|c)f(c)$, so the $(\mathbf x,c)$ pairs generated by my procedure above are distributed according to $f(\mathbf x,c)$. Marginalization amounts to ignoring the $c$'s in any pair $(\mathbf x,c)$. So the $\mathbf x$'s produced in step 2 of my procedure, taken by themselves, are distributed according to $f(\mathbf x)$, as desired.

Is this sensible? Is there a short way to make it rigorous if so? Thanks!

• what you say makes sense. But if you want to sample $c$ from the posterior distribution $[c|x]$, then your strategy won't work. – suncoolsu Jul 19 '11 at 3:27
• Oops, I think I used the wrong terminology. I meant "marginal". I'll update the question. – Mhc Jul 19 '11 at 3:35
• Oh! I almost wrote the wrong reply then! – suncoolsu Jul 19 '11 at 3:44

The formulation you describe has a general name to it. The theorem is called de-Finetti's theorem for exchangeable sequences, and is the fundamental theorem behind the Bayesian philosophy and ideas. Specifically, $x_i$s are exchangeable, that is, any permutation of $x_i$s will have the same distribution as $f(x_1,\ldots,x_n)$.
In your strategy, I would be a bit more comfortable if you had drawn $c$ for every $i$. That is, generate $c$ from $f(c)$ and then generate $x_i$, and repeat this for $i=1\ldots n$. But I think if you generate enough $\mathbf {x}$s, this should not be a problem.