For multiple linear regression let $\beta$ be the true value of the coefficient such that $y_{i}=x_{i}^{T}\beta+\epsilon_{i}$ and let $\hat{\beta}$ be the estimated value found through minimizing least square error. Now we know that $E(\hat{\beta})=\beta$ but I am not sure over which probability distribution this expectation is calculated on - is it calculated over the probability distribution of $\hat{\beta}$ ?. How does one find the probability distribution of $\hat{\beta}$ given $\epsilon\sim\mathcal{N}(0,\sigma^{2})$ ? For that matter how does one caluclate the probability distribution of other variables $y,x,\beta$ ? What is the expectation of $\hat{\beta}$ calculated over the probability distribution of $\epsilon$, $E_{\epsilon}(\hat{\beta})$ ?

  • 1
    $\begingroup$ I don't see how $E(\hat{\beta})=0$ is correct. If $\beta$ is non zero then surely the expectation of its estimate is non-zero. Also you are dealing with multiple linear regression so $\beta$ is a vector not a scalar. $\endgroup$
    – Hugh
    Oct 30, 2017 at 10:45
  • $\begingroup$ Sorry for the error. It should have been $E(\hat{\beta})=\beta$. I understand $\beta$ is random vector but I am not sure how would that change the question $\endgroup$ Oct 30, 2017 at 11:05

1 Answer 1


There is a concept named Asymptotic Normality. In our case means that over high enough number of estimations, the estimators distribution is normal.
Technically this derives: $\hat{\beta} \sim \mathbf{N}(\beta, \sigma _{\hat{\beta}}^{2})$.

This PDF explains this nicely.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.