Bayesian model selection: picking the MAP model by integrating

Question

On p. 159 of Murphy's book Machine Learning: A Probabilistic Perspective, he is discussing model selection:

A more efficient approach [to model selection] is to compute the posterior over models

$$p(m \mid \mathcal{D}) = \frac{p(\mathcal{D} \mid m) p(m)}{\sum_{m \in \mathcal{M}}p(m, \mathcal{D})}$$

From this, we can easily compute the MAP model, $\hat{m} = \operatorname{argmax}p(m \mid \mathcal{D})$. This is called Bayesian model selection.

If we use a uniform prior over models, $p(m) \propto 1$, this amounts to picking the model that maximizes

$$p(\mathcal{D} \mid m) = \int p(\mathcal{D} \mid \boldsymbol{\theta})p(\boldsymbol{\theta} \mid m) d \boldsymbol{\theta}$$

I'm a bit confused about $p(\boldsymbol{\theta}\mid m)$ on the last line. If this doesn't depend on $\mathcal{D}$, then how are we getting $\boldsymbol{\theta}$? Are they saying that we have different priors on $\boldsymbol{\theta}$ for various models?

Answer 1

In the last line $p(\theta|m)$ is the prior with model $m$.

For instance, in a linear regression case, we might have two models, one with $n$ predictors and one with $n+1$ predictors. So the parameter vector $\theta$ would actually have a different length for each of those models. So there is a missing index $\theta^{(m)}$, which may be creating some confusion.