# Calculating variance of OLS estimator with correlated errors due to repeated measurements

Suppose there are $I$ people; half are male, half are female. We have measurements $y_{it}$ for individual $i$ at time $t \in \{0,1\}$, so there are two measurements per person, which are correlated.

Suppose the data are generated by the model

$$y_{it} = \beta_1+\beta_2 x_i + \epsilon_{it},$$

where $x_i \in \{0,1\}$ is a variable indicating person $i$'s gender (1 if female) and $\epsilon_{it}$ is an error term for which

$E[\epsilon_{it}]=0,$ for all $i,t$,

$\text{var}(\epsilon_{it})=E[\epsilon_{it}^2]=\sigma^2 ,$ for all $i,t$

$\text{cov}(\epsilon_{i0},\epsilon_{i1})=E[\epsilon_{i0}\epsilon_{i1}]=\sigma_{01} ,$ for all $i$, and zero otherwise. Let $\rho=\sigma_{01}/\sigma^2$ denote the correlation. The covariance matrix is then (for the observations grouped by person)

$$\Omega= \begin{pmatrix} \sigma^{2} & \sigma_{01} & 0 & 0 & \cdots &0&0\\ \sigma_{01} & \sigma^{2} & 0 & 0& \cdots & 0&0\\ 0 & 0 & \sigma^{2} & \sigma_{01}& \cdots &0&0\\ 0 & 0 & \sigma_{01} & \sigma^{2}& \cdots &0&0\\ \vdots &\vdots & \vdots & \vdots & \hspace{0em} \ddots & \vdots& \vdots\\ 0 & 0 &0 & 0 & \ldots &\sigma^{2} &\sigma_{01} \\ 0 & 0 &0 & 0 & \ldots &\sigma_{01}&\sigma^{2} \end{pmatrix}$$

Say we estimate $\beta_1$ and $\beta_2$ simply using OLS. In this case, OLS estimates $\hat\beta_1$ and $\hat\beta_2$ should remain unbiased, but their variance is not equal to $(X'X)^{-1}\sigma^2$ (with $X$ the $2I×2$ design matrix) like in standard OLS without correlated errors (serial correlation).

How to obtain the theoretical variance of the OLS estimator $\hat\beta_2$, expressed as a function of $X_i$, $\sigma^2$ and $\rho$?

Adapted from: Dunlop, Dorothy D. "Regression for longitudinal data: A bridge from least squares regression." The American Statistician 48.4 (1994): 299-303.

Note that paper has the answer (on the first page); I'm looking for the explanation.

## 1 Answer

The covariance matrix of the estimator that I derived is

$$(X^TX)^{-1} X^T\Omega X (X^TX)^{-1}$$

It can be derived by like this: $$\hat{\beta}= (X^TX)^{-1}(X^Ty)$$ $$E[\hat{\beta}]= (X^TX)^{-1}(X^TX\beta_{true})$$ $$\hat{\beta}-E[\hat{\beta}]= (X^TX)^{-1}X^T(y-X\beta_{true})=(X^TX)^{-1}X^T\epsilon$$ $$Cov(\hat{\beta})= E[(\hat{\beta}-E[\hat{\beta}])(\hat{\beta}-E[\hat{\beta}])^T]$$ $$Cov(\hat{\beta})= E[ (X^TX)^{-1}X^T \epsilon \epsilon^T X (X^TX)^{-1}]$$ $$Cov(\hat{\beta})= (X^TX)^{-1}X^T \Omega X (X^TX)^{-1}$$

Also if needed $X^T \Omega X$ can be easily rewritten in terms of $X_i$ $\sigma^2$ and $\rho$.

• This is indeed the beginning of the answer, a derivation of the theoretical Newey-West standard errors. I was having a hard time in rewriting it in terms of Xi σ2 and ρ. But I got it now; I made a stupid mistake... if anyone wants the calculations I'm happy to provide them. But try for yourself first. Like sega_sai says, it's easy :) – Nick Oct 28 '16 at 14:21