Pattern recognition and machine learning (Bishop) - Derivation of Evidence approximation I'm reading section 3.5 of the PRML book, entitled Evidence approximation, and is having difficulty understanding this part: . I don't understand how to derive (3.75) from (3.74). The author says it is because alpha and beta are sharply peaked but I don't see how it's relevant here, or is it because they are sharply peaked that the probability P(alpha, beta|t) somehoww turn into a Dirac delta function ? Thank you very much
 A: Indeed the assumption is that $p(\alpha,\beta|t)\approx \delta(\alpha-\hat{\alpha})\delta(\beta-\hat{\beta})$. 
The point is that otherwise the maximization with respect to $\alpha,\beta$ is intractable. The other extreme is when $p(\alpha,\beta)$ is approximately uniform in $\alpha,\beta$. In this case you can write $p(\alpha,\beta|t)=\frac{p(t|\alpha,\beta)p(\alpha,\beta)}{p(t)}$ from which you can maximize $p(t|\alpha,\beta)$ instead (for example in a linear basis model).
A: The approximation has nothing to do with Dirac delta function or uniformity of hyperpriors. Here the assumption is only "sharply peaked". Due to this assumption, the integrand $p(t|{\bf w},\beta)p({\bf w}|{\bf t},\alpha,\beta)$ does not change too much on the support of $p(\alpha,\beta|{\bf t})$ ("support" means the region where this pdf is not zero), so we can approximately think this integrand remains constant at $(\hat\alpha,\hat\beta)$, i.e., $p(t|{\bf w},\hat\beta)p({\bf w}|{\bf t},\hat\alpha,\hat\beta)$. This constant can be taken out of the inner integral with regard to $\alpha$ and $\beta$ to the outer integral with regard to $\bf w$. After that, what is left in the inner integral is only $\int p(\alpha,\beta|{\bf t})d\alpha d\beta$. Since $p(\alpha,\beta|{\bf t})$ is a pdf, it must integrate to 1 no matter it is a Dirac delta function or a uniform pdf. Now what left in the outer integral is the left side of (3.75).
