3
$\begingroup$

Is there an error in the reasoning in these lecture notes from a linear models course at EPFL? The author aims to establish that under the standard fixed design linear regression model with gaussian errors, the coefficient estimate $\hat{\beta}$ is independent of the residual variance estimate $S^2$. He writes,

"If $e=y-X\hat{\beta}$ is independent of $\hat{y}=X\hat{\beta}$, then $S^2=e^Te/(n-p)$ will obviously be independent of $\hat{\beta}$. Now notice that: ..."

And he goes on to show that $e$ is independent of $\hat{y}$ in the usual way, one is a projection onto the column space of $X$, the other is the projection onto its orthogonal complement, and the distributions are jointly gaussian so uncorrelatedness implies independence.

But I don't follow what he suggests is "obvious", i.e., why $e$ being independent of $X\hat{\beta}$ implies $e$ is independent of $\hat{\beta}$, since $X$ need not be invertible. In fact, while it is easy to visualize the independence of $e$ and $\hat{y}$ in terms of projections, I don't know how to visualize the relationship between $e$ and $\hat{\beta}$, the projection scaled by the squared norm of $X$, in a way that suggests uncorrelatedness.

$\endgroup$
2
  • 1
    $\begingroup$ $\hat \beta$ is a function of $\hat Y = X \hat \beta$, since $\hat \beta = (X'X)^{-1} X'\hat Y$. $\endgroup$ Commented Aug 23, 2020 at 18:47
  • $\begingroup$ @BigBendRegion oh right, betahat must be a function of yhat when the columns of X are independent. $\endgroup$
    – Hasse1987
    Commented Aug 23, 2020 at 22:00

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.