4
$\begingroup$

According to Gauss-Markov Theorem, ordinary least squares (OLS) is the best linear unbiased estimator (BLUE). How then can restricted OLS have less variance? Please tell me the reason.

$\endgroup$
  • $\begingroup$ I proposed an edit to your question. If your edit does not address your question please correct it. $\endgroup$ – Ferdi Oct 15 '16 at 16:39
  • $\begingroup$ No entirely sure but if OLS assumption of no correlation not met, but GLS correlation assumption met then possibly restricted OLS has the lesser variance. $\endgroup$ – Carl Oct 15 '16 at 16:54
  • $\begingroup$ Please explain why you think the variance is smaller with the restrictions! $\endgroup$ – whuber Oct 15 '16 at 17:10
  • 3
    $\begingroup$ I suppose the OP thinks so because of the following sentence "Consequently, the diagonal elements of $ V(\hat{\beta}_{R})$ (variances of each $ \hat{\beta}_{Rj}$) are equal to or less than the corresponding elements of $ V(\hat{\beta})$ (variances of each $ \hat{\beta}_{j})$. This means that, if the a priori information is correct (as we will show later , in this case $ \textrm{E}(\hat{\beta}_{R})=\beta)$, the estimator vector $ \hat{\beta}_{R}$ is more efficient than the OLS estimator." It's in the reference he linked fedc.wiwi.hu-berlin.de/xplore/tutorials/xegbohtmlnode18.html $\endgroup$ – Pere Oct 15 '16 at 23:17
3
$\begingroup$

The main intuition is that restricted OLS are generally biased. So there is a tradeoff between bias and variance: you reduce variance but you allow bias.

  • An example: suppose that you want to estimate the average height of the people in your state. You may have learned that if you have a random sample (say of 2000 individuals), a "reasonable" estimator is the sample average, which is unbiased. But you have "prophecy" skills and you know for sure that the average of the population is 175cm. Then there is no variance at all, it is zero, which is lower than any estimator you can come up with. But, except if you are a really good prophet (or your cheated with data) it is likely to be biased.

  • A more explicit answer to your question would be a direct comparison of the variances of the restricted and unrestricted estimator.

$Var(\beta_u) = \sigma^2(X'X)^{-1}$

$Var(\beta_c) = \sigma^2(X'X)^{-1} - \sigma^2(X'X)^{-1}R'(R(X'X)^{-1}R')^{-1}R(X'X)^{-1}$

Therefore the variance of the restricted estimator is always weakly smaller than the variance unrestricted estimator, with equality when the restrictions are true.

  • Advanced : You can see this early test discussing the topic: the condition so that

$MSE(\hat{\beta_u}) - MSE(\hat{\beta_c})$ is positive semidefinite [$\beta_u$ is the unconstrained, $\beta_c$ the constrained estimator]

The condition is met iff $\lambda< 1/2$, where

$\lambda = \frac{1}{2\sigma^2}(R\beta-r)'(R(X'X)^{-1}R')^{-1}(R\beta-r)$, where $R\beta = r$ is the vector of contraints you impose.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.