The ABC algorithm is given as
- Draw $\theta \sim \pi(\theta)$
- Simulate data $X \sim \pi(x | \theta)$
- Accept $\theta$ if $\rho(X, D) < \varepsilon$
where $\pi(\theta)$ is the prior, $\pi(x | \theta)$ is the likelihood, $\rho(\cdot | \cdot)$ is some distance measure, $D$ is the observed data and $\varepsilon$ is the tolerance that represents a trade off between accuracy and computability.
Generally, in papers that I have seen on this, a proof is given where it states we actually sample from $\pi_{\varepsilon} = \pi(\theta | \rho(X, D) < \varepsilon)$ and then if $\varepsilon \to 0$, this converges to the true posterior $\pi(\theta | D)$.
If in Step 3, we had
3*. Accept $\theta$ if $X = D$
I was wondering if anyone knew how to prove that in this new algorithm, we sample from the true posterior? So there is no $\varepsilon \to 0$ argument?