Understand a statement about likelihood function

Question

I'm reading Agresti - Categorical Data Analysis and it says

Consider two models, $M_0$ with fitted values $\hat{\mu}_0$ and $M_1$ with fitted values $\hat{\mu}_1$ with $M_0$ a special case of $M_1$. A smaller set of parameter values satisfies $M_0$ than satisfies $M_1$. Maximizing the log likelihood over a smaller space cannot yield a larger maximum. Thus $L(\hat{\mu}_0;y) \leq L(\hat{\mu}_1;y)$

But this is not like say $L(\theta)\leq L(\hat{\theta}), \forall \theta$ if $\hat{\theta}$ is the MLE. Because in that quote, the dimensions are different. Maybe the intuition is correct: the fit is more "likely" if I use more parameters to adjust the data. But I'd like a mathematical explanation of that quote.

Thanks

Answer 1

This is only a statement of the training error.

As $M_0$ is a special case of $M_1$, each choice of parameter $\mu$ valid in $M_0$ is also valid in $M_1$(or better: has an equivalent model in $M_1$). Therefore if $\mu_0$ is the optimal solution, it is also contained in $M_1$, therefore $\mu_1=\mu_0$. If $\mu_0$ is not optimal, there might exist a parameter $\nu$ in $M_1$ with larger likelihood on the dataset and therefore $\mu_1=\nu$. This does not say anything about the generalisation error.

Answer 2

This statement is only correct if the fitted values are obtained as MLEs over the stated sets of parameter values. To see the implication in this case, suppose that $\hat{\mu}_0$ and $\hat{\mu}_1$ are MLEs over $M_0$ and $M_1$ respectively. (For simplicity of notation, I will take these as sets of parameters.) Then since $M_0 \subseteq M_1$ we have:

$$L_y(\hat{\mu}_0) = \underset{\mu \in M_0}{\max} L_y(\mu) \leqslant \max \Big( \underset{\mu \in M_0}{\max} L_y(\mu), \underset{\mu \in M_1-M_0}{\max} L_y(\mu) \Big) = \underset{\mu \in M_1}{\max} L_y(\mu) = L_y(\hat{\mu}_1).$$