What is the variance of the estimator in ordinary least squares with correlated residuals

Question

If we assumed that $y \sim N(X\beta,S)$ where

S= $\sigma^2\begin{bmatrix} 1 & \rho & \rho &...\\ \rho & 1 & \rho &...\\ \rho & \rho & 1 &...\\ \rho & \rho & \rho & ...\\ ... & ... & ... & ...\\ \end{bmatrix}$

Then the OLS approximation of $\beta$ would be the same as in the independent case:

$\hat{\beta} = (X'X)^{-1}X'y$.

However, the distribution of $\hat{\beta}$ seems to become harder to determine.

The variance of $\hat{\beta}$ would be $Var((X'X)^{-1}X'y)$ = $(X'X)^{-1}X'SX(X'X)^{-1}$. How can I further simplify this?

Answer 1

(On a side note, this is a case of correlated errors -residuals are always correlated).

You are considering a very specific form of correlation, equicorrelation. Then note that

$$S=\sigma^2[(1-\rho)I+\rho\mathbf i \mathbf i']$$

where $I$ is the identity matrix and $\mathbf i = (1,...,1)'$.

It follows that

$$\text{Var}\left(\hat \beta_{OLS}\mid X\right) = \sigma^2(X'X)^{-1}X'\big[(1-\rho)I_p+\rho\mathbf i \mathbf i'\big]X(X'X)^{-1}$$

$$=(1-\rho)\sigma^2(X'X)^{-1}X'X(X'X)^{-1} + \rho\sigma^2(X'X)^{-1}X'\mathbf i \mathbf i'X(X'X)^{-1}$$

$$=(1-\rho)\sigma^2(X'X)^{-1} + \rho\sigma^2\Big[(X'X)^{-1}X'\mathbf i\Big] \Big[(X'X)^{-1}X'\mathbf i\Big]'$$

Further, on can show that, if the regressor matrix includes a constant term, then

$$\Big[(X'X)^{-1}X'\mathbf i\Big] = (1, \mathbf 0)'$$

so we arrive at

$$\text{Var}\left(\hat \beta_{OLS}\mid X\right) = (1-\rho)\sigma^2(X'X)^{-1} + \rho\sigma^2\cdot \left[ \begin{matrix} 1 & \mathbf 0 \\ \mathbf 0 & \mathbf 0 \end{matrix} \right]$$