# Expected value of linear regression coefficient over different probability distributions

For multiple linear regression let $\beta$ be the true value of the coefficient such that $y_{i}=x_{i}^{T}\beta+\epsilon_{i}$ and let $\hat{\beta}$ be the estimated value found through minimizing least square error. Now we know that $E(\hat{\beta})=\beta$ but I am not sure over which probability distribution this expectation is calculated on - is it calculated over the probability distribution of $\hat{\beta}$ ?. How does one find the probability distribution of $\hat{\beta}$ given $\epsilon\sim\mathcal{N}(0,\sigma^{2})$ ? For that matter how does one caluclate the probability distribution of other variables $y,x,\beta$ ? What is the expectation of $\hat{\beta}$ calculated over the probability distribution of $\epsilon$, $E_{\epsilon}(\hat{\beta})$ ?

• I don't see how $E(\hat{\beta})=0$ is correct. If $\beta$ is non zero then surely the expectation of its estimate is non-zero. Also you are dealing with multiple linear regression so $\beta$ is a vector not a scalar.
– Hugh
Oct 30, 2017 at 10:45
• Sorry for the error. It should have been $E(\hat{\beta})=\beta$. I understand $\beta$ is random vector but I am not sure how would that change the question Oct 30, 2017 at 11:05

Technically this derives: $\hat{\beta} \sim \mathbf{N}(\beta, \sigma _{\hat{\beta}}^{2})$.