# Monte Carlo computation of expectation when there is dirac delta

Let $Z \sim N(0,1)$ and let $Y=Z$. Suppose I wish to perform the following weird computation:

$f(z)=\int f(z|y)f(y)dy=E_Y[f(z|y)]$

and then use Monte Carlo to estimate $E_Y[f(z|y)]$. The problem is for a fixed $z$, the $y$ is draw is almost never equal to $z$, and so $f(z|y)$ is almost always zero, and so my Monte Carlo in practice will give me $E_Y[f(z|y)]=0$.

How should I address the fact that there is actually a probability $0$ that I will draw the much needed $y=z$ for which $f(z|y)$ is the infinite spike of strength $f(z)$, so that my Monte Carlo is actually supposed to work out?

In case you are wondering why I am doing such a weird "exercise", my actual context involves some knowledge of Dirichlet inference. I am trying to do something similar to slide 44 of this presentation, except with infinitely-many clusters. Basically, in a similar spirit to that slide, I would like

$f(\theta_{n+1} |X_1,...,X_n) \\ =\int f(\theta_{n+1} |\theta_1,...,\theta_nX_1,...,X_n) f(\theta_1,...,\theta_n|X_1,...,X_n) dX_1...dX_n \\ =E[f(\theta_{n+1} | \theta_1,...,\theta_n)]$

where $\theta_1,...,\theta_n \sim \theta_1,...,\theta_n | X_1,...,X_n$

I perfectly know how to use Gibbs Sampling to sample from $\theta_1,...,\theta_n | X_1,...,X_n$. However, I am worried about the fact that $f(\theta_{n+1} | \theta_1,...,\theta_n)$ contains dirac delta functions. Same concern as above: for a fixed $\theta_{n+1}$, I will almost never draw some $\theta_i$ equal to $\theta_{n+1}$. And if the dirac delta functions never come into play in my Monte Carlo, then $E[f(\theta_{n+1} | \theta_1,...,\theta_n)]$ with Monte Carlo will just be $h(\theta_{n+1})$ where $h$ was the distribution for the $DP$ - that is, no updating happened!!

Once you have a posterior sample, $\theta_1,\dots,\theta_N$, of the parameters of a model with density $f(\cdot\vert \theta)$ and a certain prior, this is a sample from $\theta\vert \text{Observations}$, then you can approximate the predictive distribution by:
$$\pi_{pred}(x_{n+1}\vert Observations)\approx\dfrac{1}{N}\sum_{j=1}^N f(x\vert\theta_j)$$
• Hi, but what I really want is $\pi_{pred}(\theta_{n+1} | Observations)$ because $\theta$ is really my variable of interest and the $X$'s are just noisy observation of $\theta$'s. (same context as the slide 44) Aug 1, 2014 at 11:45