Independence between variance and regression coefficients In a regression model with 3 regressors, how can I prove that $\hat{\beta_1}-\hat{\beta_3}$ and $s^2$ are statistically independent? My idea is to start with $Cov((\hat{\beta_1}-\hat{\beta_3}), s^2)=Cov(\hat{\beta_1},s^2)-Cov(\hat{\beta_3},s^2)$ but I can't calculate $Cov(\hat{\beta_1},s^2), Cov(\hat{\beta_3},s^2)$.
 A: $\newcommand{\Cov}{\mathrm{Cov}}$ Calculate the analytical covariance matrix of the estimator for the parameters and see if the corresponding off-diagonal elements are zero. Then you know at least if the parameters are correlated or not, but not necessarily independent. If a Gaussian distribution defines the estimator, then off-diagonal elements equal to zero is equivalent to independence.
For example, consider the linear model:
$$
y = H\theta + \epsilon
$$
with $\epsilon \sim \mathcal{N}(0,Q)$. The least-square estimator is
$$
\hat{\theta} = (H^\top Q^{-1} H)^{-1}H^\top Q^{-1}y
$$
with covariance matrix
$$
\Cov(\hat{\theta}) = (H^\top Q^{-1} H)^{-1}.
$$
The off-diagonal elements of the above matrix then define the correlation between the parameters in $\theta$.
A: Let $y=X\beta+\epsilon$ and let $\hat{\beta}=(X^TX)^{-1}X^Ty$ be the OLS estimator. Denote $P_X$ as the projection matrix of $X$, we can then write the residuals as:
$$e=y-\hat{y}=y-X\hat{\beta}=y-X(X^TX)^{-1}X^Ty=(I_n-P_X)y=Qy$$
Where $I_n$ is $n$-size identity matrix and $Q=I_n-P_X$. Note that $Q$ is also a projection matrix and thus is idempotent.
Easily, using $e=Qy$ we can show that $e=Q\epsilon$ and $e^Te=\epsilon^TQ\epsilon$:
$$e=Qy=Q(X\beta+\epsilon)=QX\beta+Q\epsilon=(I_n-P_X)X\beta+Q\epsilon=X\beta-X\beta+Q\epsilon=Q\epsilon$$
$$e^Te=(Q\epsilon)^T(Q\epsilon)=\epsilon^TQ^TQ\epsilon=\epsilon^TQ\epsilon$$
(the last transition is due to the idempotence of $Q$).
Now, let's look at the covariance of $\hat{\beta}$ and $s^2$:
$$Cov(\hat{\beta},s^2)=Cov((X^TX)^{-1}X^Ty,Q\epsilon)$$
using $Cov(Au,Bv)=A\cdot Cov(u,v)\cdot B^T$ we get:
$$Cov((X^TX)^{-1}X^Ty,Q\epsilon)=(X^TX)^{-1}X^TCov(y,\epsilon)Q^T$$
Now, as $e=Qy=Q\epsilon$ we can write $y=(Q^TQ)^{-1}Q^Te=(Q^TQ)^{-1}Q^TQ\epsilon$ and proceed:
$$(X^TX)^{-1}X^TCov(y,\epsilon)Q^T=(X^TX)^{-1}X^TCov((Q^TQ)^{-1}Q^TQ\epsilon,\epsilon)Q^T=(X^TX)^{-1}X^T\left(\underset{=I}{(Q^TQ)^{-1}Q^TQ}\right)Cov(\epsilon,\epsilon)Q^T=\sigma^2(X^TX)^{-1}X^TQ^T=\sigma^2(X^TX)^{-1}X^TQ=\sigma^2(X^TX)^{-1}X^T(I_n-P_X)=\sigma^2((X^TX)^{-1}X^T-(X^TX)^{-1}X^TX(X^TX)^{-1}X^T)\\=\sigma^2((X^TX)^{-1}X^T-(X^TX)^{-1}X^T)=0$$
We get that $\hat{\beta}$ and $s^2$ are independent, and so are $s^2$ and $\hat{\beta}_1-\hat{\beta}_3$. $\blacksquare$
