Proof that Regression Sum of Squares and Residual Sum of Squares are independent random variables

Question

Having consulted a number of sources, I still can't find a complete proof that Regression Sum of Squares ($SS_{regression}$) and ($SS_{residual}$) are independent random variables. I'll be doubly pleased with a proof in matrix form. If it is too involved to be typed up here, I am happy to check it out myself if people can direct me to a book that contains the proof.

Answer 1

Assume $y \sim \operatorname{Normal}(\beta X, \Sigma)$ with constant diagonal covariance $\operatorname{Cov}(y)=\Sigma=\sigma^2 \mathbb I$ and mean $\bar y=\mu$. Using the hat matrix $\mathbb H$ we have that:

$$\hat y =\mathbb H y$$

And

$$\epsilon=y-\hat y=(\mathbb I -\mathbb H)y$$

Then (because the hat matrix is idempotent $\mathbb H^2 = \mathbb H$) $$\begin{cases} SSR = (\hat y - \mu)^T(\hat y - \mu)= y^T \mathbb Hy \color{red}{+n\mu^2-2\mu\mathbf 1^T\hat y}= y^T \mathbb Hy \color{red}{-n\mu^2}\\ SSE = \epsilon ^T \epsilon = y^T (\mathbb I -\mathbb H)y \end{cases}$$

$$\operatorname{Cov}(SSR,SSE) = \operatorname{Cov}(y^T \mathbb Hy,y^T (\mathbb I -\mathbb H)y)$$

Using that $\operatorname{Cov}(x^TAx,x^TBx)=4\mu^TA\Sigma B\mu + 2\operatorname{tr}(A\Sigma B\Sigma)$ (see Prove that $\mathrm{Cov}(x^TAx,x^TBx) = 2 \mathrm{Tr}(A \Sigma B \Sigma) + 4 \mu^TA \Sigma B \mu$)

$$\operatorname{Cov}(SSR,SSE) =4\mu^T\mathbb H\Sigma (\mathbb I -\mathbb H)\mu + 2\operatorname{tr}(\mathbb H\Sigma (\mathbb I -\mathbb H)\Sigma)\\ $$

Since $\Sigma=\sigma^2 \mathbb I$ is constant diagonal:

$$\operatorname{Cov}(SSR,SSE) =4\sigma^2\mu^T\mathbb H (\mathbb I -\mathbb H)\mu + 2\sigma^4\operatorname{tr}(\mathbb H (\mathbb I -\mathbb H))\\ =4\sigma^2\mu^T (\mathbb H -\mathbb H^2)\mu +2\sigma^4\operatorname{tr}(\mathbb H -\mathbb H^2)\\ =0$$