# Statistical independence of least square estimator and residual in multiple linear regression

I'm currently self studying linear regression. Following is an entrance exam problem of a graduate school. Consider the regression model with usual assumptions of the errors $y=X\beta+\epsilon$. Show that $\hat{β}$ and $e$ are statistically independent. $\hat{β}$ is the least square estimator and $e$ is the residual.) Honestly I don't even heard about independence of two random vectors. (I searched my text book of mathematical statistics and linear regression but I couldn't.) So how should I approach it? Thanks in advance. Source of the problem is graduate entrance exam of Seoul National University of 2014

• Please add the [self-study] tag & read its wiki. Feb 26 '15 at 5:23
• Is $\epsilon$ multivariate normal? What about $\hat{\beta}$? What do you know about independence of components of a multivariate normal random vector and the covariance matrix? Can you compute the covariance between $e$ and $\hat{\beta}$? Can you at least get $\hat{\beta}$ and $e$ into the same equation? Feb 26 '15 at 5:51

1. Two Random vectors $\textbf{x},\textbf{y}$ are independant iff $\forall i,j \in 1...n$, $\textbf{x}_{i}$ and $\textbf{y}_{j}$ are independant.
2. Cross covariance matrix is defined by $cov(x,y )= E[(\textbf{x} - \mu_{\textbf{x}})(\textbf{y} - \mu_{\textbf{y}})^{T}]$
4. under the assumption that distribution of $\epsilon$ is $N(0,\sigma I)$ and $\widehat{\beta}$ is the ordinary least square esitimator $Hy$, if one compute covariance matrix of $\widehat{\beta}$ and $\textbf{e}$ , it is zero. Therefore they are statistically independant.