Consider a training dataset $X$, a probabilistic model parameterized by $\theta$, and a prior $P(\theta)$. For a new data point $x^*$, we can compute $P(x^*)$ using:
- a fully bayesian approach: the posterior predictive distribution $P(x^* | X) = \int P(\theta|X) P(x^*|\theta) d\theta$
- the likelihood parameterized by the maximum a posteriori estimate: $P(x^* | \theta_{MAP})$, where $\theta_{MAP} = \text{argmax}_\theta P(\theta|X)$
Is the fully bayesian approach always "better" than the MAP approach? More precisely, is the MAP approach an approximation of the bayesian approach, in the sense that we are we hoping that $P(x^* | \theta_{MAP})$ is a good approximation of $P(x^* | X)$?