Say we have a full regression model \begin{align*} \mathbf{y} &= \mathbf{X} \boldsymbol{\beta} + \boldsymbol{\epsilon}\\ &= \mathbf{X}_p \boldsymbol{\beta}_p + \mathbf{X}_r \boldsymbol{\beta}_r + \boldsymbol{\epsilon}\\ \end{align*} and a smaller model $$ \mathbf{y} = \mathbf{X}_p \boldsymbol{\beta}_p + \widetilde{\boldsymbol{\epsilon}}, $$ where both $\boldsymbol{\epsilon}$ and $\widetilde{\boldsymbol{\epsilon}}$ have the covariance matrix $\sigma^2 \mathbf{I}$. Note that, if the full model is true, $\widetilde{\boldsymbol{\epsilon}} = \mathbf{X}_r \boldsymbol{\beta}_r + \boldsymbol{\epsilon}$
The estimate of $\boldsymbol{\beta}_p$ from the full model is $$ \hat{\boldsymbol{\beta}}_p^* = \left[(\mathbf{X}^\intercal\mathbf{X})^{-1} \mathbf{X}^\intercal \mathbf{y}\right]_p $$ and the estimate from the smaller model is $$ \hat{\boldsymbol{\beta}}_p = (\mathbf{X}_p^\intercal\mathbf{X}_p)^{-1} \mathbf{X}_p^\intercal \mathbf{y}. $$
The bias for the full model is $\mathbf{0}$, and the bias for the small model is $$ (\mathbf{X}_p^\intercal \mathbf{X}_p)^{-1} \mathbf{X}_p^\intercal \mathbf{X}_r \boldsymbol{\beta}_r = \mathbf{A}\boldsymbol{\beta}_r . $$
The covariance matrix for the full model's estimates is $$ V[\hat{\boldsymbol{\beta}}^*_p] = \sigma^2 \left[(\mathbf{X}^\intercal\mathbf{X})^{-1}\right]_p $$ and the covariance matrix for the smaller model's estimates is $$ V[\hat{\boldsymbol{\beta}}_p] = \sigma^2 (\mathbf{X}_p^\intercal\mathbf{X}_p)^{-1}. $$ Note that the difference between these is positive semi-definite by the formula for inverses of block matrices.
What are all of the situations where $\text{MSE}[\hat{\boldsymbol{\beta}}^*_p] - \text{MSE}[\hat{\boldsymbol{\beta}}_p]$ is positive semi-definite?
- One is when $\mathbf{X}_p^\intercal \mathbf{X}_r = 0$. Then both biases are zero, and $$ V[\hat{\boldsymbol{\beta}}^*_p] - V[\hat{\boldsymbol{\beta}}_p] $$ is positive semidefinite because, as Yves points out, the variances are the same because the estimates are the same.
Are there any others? This book mentions that it is also true when $$ V[\hat{\boldsymbol{\beta}}^*_r] - \boldsymbol{\beta}_r\boldsymbol{\beta}_r^\intercal $$ is positive semi-definite, but I haven't been able to show this.