# Why does the first term in this marginal likelihood approximation decrease with model complexity

In Bishop's PRML chapter 3, he presents the following approximation for the marginal likelihood in equation 3.72

$$\ln P(D) \approx \ln P(D| \boldsymbol{w}_{map}) + \ln M\frac{\Delta w_{posterior}}{\Delta w_{prior}}$$

Where $$D$$ is the data, and $$\boldsymbol{w}_{map}$$ is a vector of model parameters around which there is a peak in the posterior. This is all dependent one of several models $$M_i$$, but that notation is dropped to reduce clutter

We can ignore the second term on the right-hand side for now. I want to understand something he says about the first term, $$P(D|\boldsymbol{w}_{map})$$

First, he notes that for a flat prior, the $$\ln{P(D|\boldsymbol{w}_{map})}$$ corresponds to the log-likelihood, which makes sense (although I don't see the dependence on the prior's shape).

My trouble starts with his second claim about $$\ln{P(D|\boldsymbol{w}_{map})}$$. He states

"As we increase the complexity of the model, the first term will typically decrease, because a more complex model is better able to fit the data..."

Why would the likelihood decrease with a better fit? The log is a monotonic increasing function of its input and my understanding of the likelihood is that it will increase with better fit. Why then, as we add parameters and increase the complexity of the model, would we see the likelihood decrease?

Thanks

This is a typo: the first term increases with $$M$$ and the second term decreases with $$M$$ since $$\frac{\Delta w_{posterior}}{\Delta w_{prior}}<1$$ In the Errata published in 2011 by Svensén & Bishop, the exchange between increasing and decreasing is acknowledged: