Suppose there are $I$ people; half are male, half are female. We have measurements $y_{it}$ for individual $i$ at time $t \in \{0,1\}$, so there are two measurements per person, which are correlated.

Suppose the data are generated by the model

$$y_{it} = \beta_1+\beta_2 x_i + \epsilon_{it},$$

where $x_i \in \{0,1\}$ is a variable indicating person $i$'s gender (1 if female) and $\epsilon_{it}$ is an error term for which

$E[\epsilon_{it}]=0,$ for all $i,t$,

$\text{var}(\epsilon_{it})=E[\epsilon_{it}^2]=\sigma^2 ,$ for all $i,t$

$\text{cov}(\epsilon_{i0},\epsilon_{i1})=E[\epsilon_{i0}\epsilon_{i1}]=\sigma_{01} ,$ for all $i$, and zero otherwise. Let $\rho=\sigma_{01}/\sigma^2$ denote the correlation. The covariance matrix is then (for the observations grouped by person)

$$ \Omega= \begin{pmatrix} \sigma^{2} & \sigma_{01} & 0 & 0 & \cdots &0&0\\ \sigma_{01} & \sigma^{2} & 0 & 0& \cdots & 0&0\\ 0 & 0 & \sigma^{2} & \sigma_{01}& \cdots &0&0\\ 0 & 0 & \sigma_{01} & \sigma^{2}& \cdots &0&0\\ \vdots &\vdots & \vdots & \vdots & \hspace{0em} \ddots & \vdots& \vdots\\ 0 & 0 &0 & 0 & \ldots &\sigma^{2} &\sigma_{01} \\ 0 & 0 &0 & 0 & \ldots &\sigma_{01}&\sigma^{2} \end{pmatrix} $$

Say we estimate $\beta_1$ and $\beta_2$ simply using OLS. In this case, OLS estimates $\hat\beta_1$ and $\hat\beta_2$ should remain unbiased, but their variance is not equal to $(X'X)^{-1}\sigma^2$ (with $X$ the $2I×2$ design matrix) like in standard OLS without correlated errors (serial correlation).

How to obtain the theoretical variance of the OLS estimator $\hat\beta_2$, expressed as a function of $X_i$, $\sigma^2$ and $\rho$?

Adapted from: Dunlop, Dorothy D. "Regression for longitudinal data: A bridge from least squares regression." The American Statistician 48.4 (1994): 299-303.

Note that paper has the answer (on the first page); I'm looking for the explanation.


The covariance matrix of the estimator that I derived is

$$ (X^TX)^{-1} X^T\Omega X (X^TX)^{-1}$$

It can be derived by like this: $$\hat{\beta}= (X^TX)^{-1}(X^Ty)$$ $$E[\hat{\beta}]= (X^TX)^{-1}(X^TX\beta_{true})$$ $$\hat{\beta}-E[\hat{\beta}]= (X^TX)^{-1}X^T(y-X\beta_{true})=(X^TX)^{-1}X^T\epsilon$$ $$Cov(\hat{\beta})= E[(\hat{\beta}-E[\hat{\beta}])(\hat{\beta}-E[\hat{\beta}])^T]$$ $$Cov(\hat{\beta})= E[ (X^TX)^{-1}X^T \epsilon \epsilon^T X (X^TX)^{-1}]$$ $$Cov(\hat{\beta})= (X^TX)^{-1}X^T \Omega X (X^TX)^{-1}$$

Also if needed $X^T \Omega X$ can be easily rewritten in terms of $X_i$ $\sigma^2$ and $\rho$.

  • $\begingroup$ This is indeed the beginning of the answer, a derivation of the theoretical Newey-West standard errors. I was having a hard time in rewriting it in terms of Xi σ2 and ρ. But I got it now; I made a stupid mistake... if anyone wants the calculations I'm happy to provide them. But try for yourself first. Like sega_sai says, it's easy :) $\endgroup$ – Nick Oct 28 '16 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.