Restricted OLS have less variance than OLS? According to Gauss-Markov Theorem, ordinary least squares (OLS) is the best linear unbiased estimator (BLUE). How then can restricted OLS have less variance?
Please tell me the reason.
 A: The main intuition is that restricted OLS are generally biased. So there is a tradeoff between bias and variance: you reduce variance but you allow bias.

*

*An example: suppose that you want to estimate the average height of the people in your state. You may have learned that if you have a random sample (say of 2000 individuals), a "reasonable" estimator is the sample average, which is unbiased. But you have "prophecy" skills and you know for sure that the average of the population is 175cm. Then there is no variance at all, it is zero, which is lower than any estimator you can come up with. But, except if you are a really good prophet (or your cheated with data) it is likely to be biased.


*A more explicit answer to your question would be a direct comparison of the variances of the restricted and unrestricted estimator.
$$Var(\beta_u) = \sigma^2(X'X)^{-1}$$
$$Var(\beta_c) = \sigma^2(X'X)^{-1} - \sigma^2(X'X)^{-1}R'(R(X'X)^{-1}R')^{-1}R(X'X)^{-1}$$
Therefore the variance of the restricted estimator is always weakly smaller than the variance unrestricted estimator, with equality when the restrictions are true.

*

*Advanced : You can see this early test discussing the topic: the condition so that

$MSE(\hat{\beta_u}) - MSE(\hat{\beta_c})$ is positive semidefinite [$\beta_u$ is the unconstrained, $\beta_c$ the constrained estimator]
The condition is met iff $\lambda< 1/2$, where
$$\lambda = \frac{1}{2\sigma^2}(R\beta-r)'(R(X'X)^{-1}R')^{-1}(R\beta-r),$$
where $R\beta = r$ is the vector of constraints you impose.
A: I don't think restricted OLS are biased. Look at Hansen's textbook Econometrics section 8.4 "Finite sample properties". Theorem 8.2 says CLS estimators are unbiased for $\beta$.
A: Consider $X_1 \sim N(\mu_1,1)$ and $X_2 \sim N(\mu_2,1)$. And we want to estimate $\mu_1$ and $\mu_2$.

*

*Unrestricted case. We will use $\hat{\mu}_i = X_i$ and the variance of the error will be equal to one.


*Restricted case. Let for instance impose the restriction $\mu_1+\mu_2 = 0$ or $\mu_1 = - \mu_2$. Now we will estimate a single parameter $\hat{\mu}  = \frac{X_1-X_2}{2}$ instead (and we can make the relationships $\hat{\mu}_1 = \hat{\mu}$, $\hat{\mu}_2 = -\hat{\mu}$).
The variance of the estimate will now be equal to $1/2$. Indeed, it is a smaller variance.


*Restricted case. When the restriction is untrue. If we do not truly have $\mu_1 = - \mu_2$ then our estimate $\hat{\mu} \sim N(\frac{\mu_1 - \mu_2}{2}, \frac{1}{2})$ will be biased and we may have a larger error.
The restricted case is not unbiased.
