Monte Carlo computation of expectation when there is dirac delta

Question

Let $Z \sim N(0,1)$ and let $Y=Z$. Suppose I wish to perform the following weird computation:

$f(z)=\int f(z|y)f(y)dy=E_Y[f(z|y)]$

and then use Monte Carlo to estimate $E_Y[f(z|y)]$. The problem is for a fixed $z$, the $y$ is draw is almost never equal to $z$, and so $f(z|y)$ is almost always zero, and so my Monte Carlo in practice will give me $E_Y[f(z|y)]=0$.

How should I address the fact that there is actually a probability $0$ that I will draw the much needed $y=z$ for which $f(z|y)$ is the infinite spike of strength $f(z)$, so that my Monte Carlo is actually supposed to work out?

---

In case you are wondering why I am doing such a weird "exercise", my actual context involves some knowledge of Dirichlet inference. I am trying to do something similar to slide 44 of this presentation, except with infinitely-many clusters. Basically, in a similar spirit to that slide, I would like

$f(\theta_{n+1} |X_1,...,X_n) \\ =\int f(\theta_{n+1} |\theta_1,...,\theta_nX_1,...,X_n) f(\theta_1,...,\theta_n|X_1,...,X_n) dX_1...dX_n \\ =E[f(\theta_{n+1} | \theta_1,...,\theta_n)]$

where $\theta_1,...,\theta_n \sim \theta_1,...,\theta_n | X_1,...,X_n$

I perfectly know how to use Gibbs Sampling to sample from $\theta_1,...,\theta_n | X_1,...,X_n$. However, I am worried about the fact that $f(\theta_{n+1} | \theta_1,...,\theta_n)$ contains dirac delta functions. Same concern as above: for a fixed $\theta_{n+1}$, I will almost never draw some $\theta_i$ equal to $\theta_{n+1}$. And if the dirac delta functions never come into play in my Monte Carlo, then $E[f(\theta_{n+1} | \theta_1,...,\theta_n)]$ with Monte Carlo will just be $h(\theta_{n+1})$ where $h$ was the distribution for the $DP$ - that is, no updating happened!!

Answer 1

The problem in the first part of your question is wrongly formulated given that the conditional density does not exist.

According to slide 44, what you are interested in is the Posterior Predictive Distribution, which is a well-defined object in Bayesian inference.

Once you have a posterior sample, $\theta_1,\dots,\theta_N$, of the parameters of a model with density $f(\cdot\vert \theta)$ and a certain prior, this is a sample from $\theta\vert \text{Observations}$, then you can approximate the predictive distribution by:

$$\pi_{pred}(x_{n+1}\vert Observations)\approx\dfrac{1}{N}\sum_{j=1}^N f(x\vert\theta_j)$$