# Large Sample Properties Least Square Estimator

I am trying to understand the properties of the least squares estimator. In particular the Gauss-Markov theorem states that the estimator is independent from the distribution of the error term.

Thus if I understand correctly, the following should be true:

$\beta | e \sim N(\mu, \sigma^2) == \beta|e\sim Beta(2,5)$

However when I tried out an example I got slightly different results:

#code in julia
(() -> begin
n = 10000
A = [ones(n) rand(n)]
β = [3.1, 12.7]
y = A * β + rand(Beta(2,5), n)
b = inv(A'A)A'y
print(b)
end)()

> 3.38 12.69


and

(() -> begin
n = 10000
A = [ones(n) rand(n)]
β = [3.1, 12.7]
y = A * β + rand(Normal(), n)
b = inv(A'A)A'y
print(b)
end)()

> 3.10 12.71


Now the normal distributed errors are indeed unbiased estimators, while for the Beta distribution there seems to be a bias of $3.38 - 3.10$ for the intercept which according to theory should not be there, right?

• You;d need the error term to have mean 0. The Beta(2,5) distribution doesn't have mean 0 – Glen_b Oct 15 '15 at 13:54
• thank you for the comment. Can you post this as an answer. – Vincent Oct 15 '15 at 14:14
• I probably shouldn't - one line answers aren't really acceptable. I may be able to figure out something more to add though. – Glen_b Oct 15 '15 at 16:08
• To be honest it was not a very smart question to begin with =). It makes sense that a RV with a mean not equal to zero introduces a bias. – Vincent Oct 15 '15 at 16:13
• I'll post something soon – Glen_b Oct 15 '15 at 16:31