# How to derive the covariance matrix between $\bar{y}$ and $\hat{\beta_c}$ where $\hat{\beta_c}$ is the OLS estimator of a linear model?

$$cov\begin{bmatrix}\bar{y}\\\hat{\beta}_c\end{bmatrix}=\sigma^2\begin{bmatrix}\frac{1}{n} & 0^T\\0 & (X_c^TX_c)^{-1}\end{bmatrix}$$ with $$\hat{\beta}_c=(X_c^TX)^{-1}X^T_cy$$.

I am supposed to show two estimators are independent. I know I can show that the covariance part of the matrix is 0 but I didn't learn matrix theory back in collage. I am wondering if someone could show me how the calculation is conducted so I can have a rough idea of what should be done?

Let me start from the beginning. You have a model $$y=X\beta+\epsilon=\beta_0+\beta_1x_1+\dots+\beta_px_p+\epsilon$$ where $$\epsilon\sim\mathcal{N}(0,\sigma^2I)$$, $$y\sim\mathcal{N}(X\beta,\sigma^2I)$$, and $$\hat\beta=(X^TX)^{-1}X^Ty$$. Il you center your independent variables, you get: $$y=\beta_0+\beta_1(x_1-\bar{x}_1)+\dots+\beta_p(x_p-\bar{x}_p)+\epsilon=\tilde{X}\beta+\epsilon$$ where $$\tilde{X}=(1,X_c)$$ and $$X_c$$ has typical element $$x_{ij}-\bar{x}_j$$. The estimated coefficients are: $$\hat\beta=(\hat\beta_0,\beta_c),\qquad\hat\beta_0=\bar{y},\qquad \hat\beta_c=(X_c^TX_c)^{-1}X_c^Ty$$ In general, when $$y$$ is a random vector and $$C$$ is a matrix, $$\text{cov}(Cy)=C\text{cov}(y)C^T$$. If $$\hat\beta=(X^TX)^{-1}X^Ty$$ then, since $$X^TX$$ is symmetric: \begin{align*} \text{cov}(\hat\beta)&=(X^TX)^{-1}X^T\text{cov}(y)[(X^TX)^{-1}X^T]^T \\ &=(X^TX)^{-1}X^T\sigma^2X(X^TX)^{-1}\\ &=\sigma^2(X^TX)^{-1}(X^TX)(X^TX)^{-1}=\sigma^2(X^TX)^{-1} \end{align*} Let's now consider the simpler model $$y=\beta_0+\beta_1x$$, where $$x=(x_1,x_2,x_3)=(1,2,3)$$. The $$X^TX$$ matrix is: \begin{align*} X^TX&=\begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \end{bmatrix}\begin{bmatrix} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix} =\begin{bmatrix} \sum_j 1 & \sum_j1x_{j}\\ \sum_jx_{2}^T1 & \sum_jx_{j}^Tx_{j}\end{bmatrix}\\ &=\begin{bmatrix} n & \sum_j x_j \\ \sum_j x_j & \sum_j x_j^2 \end{bmatrix}=\begin{bmatrix}3 & 6 \\ 6 & 14 \end{bmatrix} \end{align*} Its inverse is \begin{align*} (X^TX)^{-1}&=\frac{1}{n\sum_jx_j^2-\left(\sum_jx_j\right)^2} \begin{bmatrix} \sum_jx_j^2 & -\sum_jx_j \\ -\sum_jx_j & n \end{bmatrix}\\ &=\begin{bmatrix}\frac{1}{n}+\frac{\bar{x}^2}{\sum_j(x_j-\bar{x})^2} & -\frac{\sum_jx_j}{n\sum_jx_j^2-\left(\sum_jx_j\right)^2} \\ -\frac{\sum_jx_j}{n\sum_jx_j^2-\left(\sum_jx_j\right)^2} & \frac{1}{\sum_j(x_j-\bar{x})^2} \end{bmatrix} =\frac16\begin{bmatrix}14 & -6 \\ -6 & 3\end{bmatrix}=\begin{bmatrix}2.\bar{3} & -1 \\ -1 & 0.5 \end{bmatrix} \end{align*} If you replace $$X$$ with $$\tilde{X}=(1,X_c)$$, then $$\sum_jx_j=0$$ and \begin{align*} \tilde{X}^T\tilde{X}&=\begin{bmatrix} 1 & 1 & 1 \\ -1 & 0 & 1 \end{bmatrix}\begin{bmatrix} 1 & -1 \\ 1 & 0 \\ 1 & 1 \end{bmatrix}=\begin{bmatrix} 3 & 0 \\ 0 & 2\end{bmatrix}\\ (\tilde{X}^T\tilde{X})^{-1}&=\begin{bmatrix} \frac13 & 0 \\ 0 & \frac12\end{bmatrix} \end{align*} In general (see Seber & Lee, Linear Regression Analysis, John Wiley & Sons, 2003, p. 120), $$(X^TX)^{-1}=\begin{bmatrix}\frac1n+\bar{x}^TV^{-1}\bar{x} & -\bar{x}^TV^{-1} \\ -V^{-1}\bar{x} & V^{-1}\end{bmatrix}$$ where $$\bar{x}$$ is a vector of means and $$V=X_c^TX_c$$. If $$X=\tilde{X}$$, then $$\bar{x}$$ is a null vector and $$(\tilde{X}^T\tilde{X})^{-1}=\begin{bmatrix}\frac1n & 0 \\ 0 & (X_c^TX_c)^{-1}\end{bmatrix}$$ Therefore $$\hat\beta_0=\bar{y}$$ and $$\hat\beta_c$$ are uncorrelated.

HTH

PS: You can also look at Linear regression $y_i=\beta_0 + \beta_1x_i + \epsilon_i$ covariance between $\bar{y}$ and $\hat{\beta}_1$, where linear algebra is not used.