How to derive the covariance matrix between $\bar{y}$ and $\hat{\beta_c}$ where $\hat{\beta_c}$ is the OLS estimator of a linear model? $$cov\begin{bmatrix}\bar{y}\\\hat{\beta}_c\end{bmatrix}=\sigma^2\begin{bmatrix}\frac{1}{n} & 0^T\\0 & (X_c^TX_c)^{-1}\end{bmatrix}$$
with $\hat{\beta}_c=(X_c^TX)^{-1}X^T_cy$.
I am supposed to show two estimators are independent. I know I can show that the covariance part of the matrix is 0 but I didn't learn matrix theory back in collage. I am wondering if someone could show me how the calculation is conducted so I can have a rough idea of what should be done?
 A: Let me start from the beginning. You have a model $$y=X\beta+\epsilon=\beta_0+\beta_1x_1+\dots+\beta_px_p+\epsilon$$
where $\epsilon\sim\mathcal{N}(0,\sigma^2I)$, $y\sim\mathcal{N}(X\beta,\sigma^2I)$, and $\hat\beta=(X^TX)^{-1}X^Ty$.
Il you center your independent variables, you get:
$$y=\beta_0+\beta_1(x_1-\bar{x}_1)+\dots+\beta_p(x_p-\bar{x}_p)+\epsilon=\tilde{X}\beta+\epsilon$$
where $\tilde{X}=(1,X_c)$ and $X_c$ has typical element $x_{ij}-\bar{x}_j$.
The estimated coefficients are:
$$\hat\beta=(\hat\beta_0,\beta_c),\qquad\hat\beta_0=\bar{y},\qquad \hat\beta_c=(X_c^TX_c)^{-1}X_c^Ty$$
In general, when $y$ is a random vector and $C$ is a matrix, $\text{cov}(Cy)=C\text{cov}(y)C^T$. If $\hat\beta=(X^TX)^{-1}X^Ty$ then, since $X^TX$ is symmetric:
\begin{align*}
\text{cov}(\hat\beta)&=(X^TX)^{-1}X^T\text{cov}(y)[(X^TX)^{-1}X^T]^T \\
&=(X^TX)^{-1}X^T\sigma^2X(X^TX)^{-1}\\
&=\sigma^2(X^TX)^{-1}(X^TX)(X^TX)^{-1}=\sigma^2(X^TX)^{-1}
\end{align*}
Let's now consider the simpler model $y=\beta_0+\beta_1x$, where $x=(x_1,x_2,x_3)=(1,2,3)$. The $X^TX$ matrix is:
\begin{align*}
X^TX&=\begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \end{bmatrix}\begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix}
=\begin{bmatrix} \sum_j 1 & \sum_j1x_{j}\\
\sum_jx_{2}^T1 & \sum_jx_{j}^Tx_{j}\end{bmatrix}\\
&=\begin{bmatrix} n & \sum_j x_j \\ \sum_j x_j & \sum_j x_j^2 \end{bmatrix}=\begin{bmatrix}3 & 6 \\ 6 & 14 \end{bmatrix}
\end{align*}
Its inverse is
\begin{align*}
(X^TX)^{-1}&=\frac{1}{n\sum_jx_j^2-\left(\sum_jx_j\right)^2}
\begin{bmatrix} \sum_jx_j^2 & -\sum_jx_j \\ -\sum_jx_j & n \end{bmatrix}\\
&=\begin{bmatrix}\frac{1}{n}+\frac{\bar{x}^2}{\sum_j(x_j-\bar{x})^2} & -\frac{\sum_jx_j}{n\sum_jx_j^2-\left(\sum_jx_j\right)^2} \\ -\frac{\sum_jx_j}{n\sum_jx_j^2-\left(\sum_jx_j\right)^2} & \frac{1}{\sum_j(x_j-\bar{x})^2}
\end{bmatrix}
=\frac16\begin{bmatrix}14 & -6 \\ -6 & 3\end{bmatrix}=\begin{bmatrix}2.\bar{3} & -1 \\ -1 & 0.5 \end{bmatrix}
\end{align*}
If you replace $X$ with $\tilde{X}=(1,X_c)$, then $\sum_jx_j=0$ and
\begin{align*}
\tilde{X}^T\tilde{X}&=\begin{bmatrix} 1 & 1 & 1 \\ -1 & 0 & 1 \end{bmatrix}\begin{bmatrix} 1 & -1 \\ 1 & 0 \\ 1 & 1 \end{bmatrix}=\begin{bmatrix} 3 & 0 \\ 0 & 2\end{bmatrix}\\
(\tilde{X}^T\tilde{X})^{-1}&=\begin{bmatrix} \frac13 & 0 \\ 0 & \frac12\end{bmatrix}
\end{align*}
In general (see Seber & Lee, Linear Regression Analysis, John Wiley & Sons, 2003, p. 120),
$$(X^TX)^{-1}=\begin{bmatrix}\frac1n+\bar{x}^TV^{-1}\bar{x} & -\bar{x}^TV^{-1} \\
-V^{-1}\bar{x} & V^{-1}\end{bmatrix}$$
where $\bar{x}$ is a vector of means and $V=X_c^TX_c$. If $X=\tilde{X}$, then $\bar{x}$ is a null vector and
$$(\tilde{X}^T\tilde{X})^{-1}=\begin{bmatrix}\frac1n & 0 \\
0 & (X_c^TX_c)^{-1}\end{bmatrix}$$
Therefore $\hat\beta_0=\bar{y}$ and $\hat\beta_c$ are uncorrelated.
HTH
PS: You can also look at Linear regression $y_i=\beta_0 + \beta_1x_i + \epsilon_i$ covariance between $\bar{y}$ and $\hat{\beta}_1$, where linear algebra is not used.
