Let's say that I have a method of sampling from $P(X, \theta \mid Y)$. It seems intuitive that if I drew samples of $(X, \theta)$ and then simply dropped the $\theta$s, this is equivalent to drawing from $P(X \mid Y)$ directly. What is the theorem that says that I can do this?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Under suitable regularity conditions, if you have a sequence of IID random pairs $\{(X_i,Y_i)\}_{i\geq 1}$ and a nice function $g:\mathbb{R}^2\to\mathbb{R}$, since $\{g(X_i,Y_i)\}_{i\geq 1}$ is a sequence of IID random variables, the Strong Law of Large Numbers says that $$ \frac{1}{n}\sum_{i=1}^n g(X_i,Y_i) \to \mathrm{E}[g(X_1,Y_1)] \, , $$ with probability one, when $n\to\infty$. To answer your question, consider what happens if you choose $g(x,y)=x$. This argument holds in more general settings and your simulation technique is absolutely correct.
