1
$\begingroup$

I am trying to figure out why the parameter $$\begin{equation*} \hat\beta = (X^TX)^{-1}X^TY \end{equation*}$$ is normally distributed in least-squares prediction. (Where Y is a linear function plus normal noise.) All the examples I've found have said that since $$\begin{align*} \hat\beta &= (X^TX)^{-1}X^TY \\ &= (X^TX)^{-1}X^T(X\beta + \varepsilon) \\ &= \beta + (X^TX)^{-1}X^T\varepsilon \end{align*}$$ we know that $$\hat\beta-\beta \sim \mathcal{N}(0,\sigma^2 (X^TX)^{-1})$$

I can see how the mean and variance are calculated, but why is this a normal distribution?

$\endgroup$
  • 5
    $\begingroup$ property: a linear transform of a normal is a normal $\endgroup$ – Xi'an Jan 5 '17 at 5:00
  • 1
    $\begingroup$ Possible duplicate of Help clarify the implication of normality in an Ordinary Least Square (OLS) Regression $\endgroup$ – Xi'an Jan 5 '17 at 5:02
  • 1
    $\begingroup$ Also be aware that under certain regularity conditions, the distribution of $\hat{\beta}$ will be asymptotically normal as the number of observations $n \rightarrow \infty$. For the asymptotic argument, you don't need $\epsilon$ to be normal (but you do need conditions such that a central limit theorem and other asymptotic arguments apply). $\endgroup$ – Matthew Gunn Jan 5 '17 at 8:39
2
$\begingroup$

In classical statistics the parameter value $\beta$ in a linear regression model is an unknown constant. The value $\hat{\beta}$ is not a parameter - it is an estimator of the parameter, which is a function of the data. The reason this estimator is normally distributed is that it is a linear function of the underlying error vector (as written in the equation you have shown), which is normally distributed under the model assumptions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.