2
$\begingroup$

I am trying to understand the properties of the least squares estimator. In particular the Gauss-Markov theorem states that the estimator is independent from the distribution of the error term.

Thus if I understand correctly, the following should be true:

$\beta | e \sim N(\mu, \sigma^2) == \beta|e\sim Beta(2,5)$

However when I tried out an example I got slightly different results:

#code in julia 
(() -> begin
   n = 10000
   A = [ones(n) rand(n)]
   β = [3.1, 12.7]
   y = A * β + rand(Beta(2,5), n)
   b = inv(A'A)A'y
   print(b)
end)()

> 3.38 12.69

and

(() -> begin
   n = 10000
   A = [ones(n) rand(n)]
   β = [3.1, 12.7]
   y = A * β + rand(Normal(), n)
   b = inv(A'A)A'y
   print(b)
end)()

> 3.10 12.71

Now the normal distributed errors are indeed unbiased estimators, while for the Beta distribution there seems to be a bias of $3.38 - 3.10$ for the intercept which according to theory should not be there, right?

$\endgroup$
5
  • 2
    $\begingroup$ You;d need the error term to have mean 0. The Beta(2,5) distribution doesn't have mean 0 $\endgroup$ – Glen_b Oct 15 '15 at 13:54
  • $\begingroup$ thank you for the comment. Can you post this as an answer. $\endgroup$ – Vincent Oct 15 '15 at 14:14
  • $\begingroup$ I probably shouldn't - one line answers aren't really acceptable. I may be able to figure out something more to add though. $\endgroup$ – Glen_b Oct 15 '15 at 16:08
  • $\begingroup$ To be honest it was not a very smart question to begin with =). It makes sense that a RV with a mean not equal to zero introduces a bias. $\endgroup$ – Vincent Oct 15 '15 at 16:13
  • $\begingroup$ I'll post something soon $\endgroup$ – Glen_b Oct 15 '15 at 16:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.