Suppose you have 8 observations ($i=1,...,8$) from three different states (A, B, C) and you also know that observations for $i=1,2$ are from state A, for $i=3,4,5$ are from state B and for $i=6,7,8$ are from state C. You are trying to estimate parameters with a linear regression model where $\varepsilon_i$ is the error term. The assumptions on this error term are that: $E[\varepsilon_i]=0$, $V[\varepsilon]=\sigma^2$ and:

$$Cov[\varepsilon_i, \varepsilon_j]=\begin{cases} \sigma^2 \rho & \text{ if observation i comes from the same state of observation j} \\ 0 & \text{otherwise} \end{cases}$$

Now you have that:

$$\overline{\varepsilon_h}=\frac{1}{n_h} \sum_{i \in h} \varepsilon_i$$

where $h=A,B,C$

I'm asked to compute the variance-covariance matrix of $\overline{\varepsilon}$ (notated $V[\overline{\varepsilon}]$) so I've started to compute variances and covariances of $\overline{\varepsilon}$ for $h=A, \ B, \ C$.

  • $V[\overline{\varepsilon}_A]=...=\frac{\sigma^2}{n_A}$
  • $V[\overline{\varepsilon}_B]=...=\frac{\sigma^2}{n_B}$
  • $V[\overline{\varepsilon}_C]=...=\frac{\sigma^2}{n_C}$
  • $Cov[\overline{\varepsilon}_A, \overline{\varepsilon}_B]= Cov\left [ \frac{1}{n_A} \sum_{i \in A} {\varepsilon}_i, \frac{1}{n_B} \sum_{j \in B} {\varepsilon}_j \right ]=\frac{1}{n_A} \frac{1}{n_B} Cov\left [ \sum_{i = 1}^{2} \varepsilon_i, \sum_{j = 3}^{5} \varepsilon_j \right ] = \frac{1}{n_A} \frac{1}{n_B} \sum_{i = 1}^{2} \sum_{j = 3}^{5} Cov[\varepsilon_i, \varepsilon_j]\underset{i\neq j}=0$
  • $Cov[\overline{\varepsilon}_A, \overline{\varepsilon}_C]=...= 0$
  • $Cov[\overline{\varepsilon}_B, \overline{\varepsilon}_C]=...= 0$

This leads me to the following variance-covariance matrix:

$$V[\overline{\varepsilon}]=\sigma^2\begin{bmatrix} \frac{1}{n_A} &0 &0 \\ 0 & \frac{1}{n_B} &0 \\ 0 & 0 & \frac{1}{n_C} \end{bmatrix}=\sigma^2 \begin{bmatrix} \frac{1}{2} &0 &0 \\ 0 & \frac{1}{3} &0 \\ 0 & 0 & \frac{1}{3} \end{bmatrix}$$

which apparently is not the right one. I think there is something wrong with the covariances computed above, but I can't see what. Can you please help me?


Your assumptions in the beginning are: $$ Cov[\epsilon_i,\epsilon_j] = \sigma^2\rho $$ For the variables that correspond to observations from the same state.

Now when you calculate variance for each of the three components in the end, you have to take that into account, i.e. $$ V[\overline{\epsilon}_A] = V[\frac{1}{n_A}\sum_{i=1}^2\epsilon_{iA}] $$ $$ = \frac{1}{n_A^2}\sum_{i=1}^2\sum_{j=1}^2Cov(\epsilon_{iA},\epsilon_{jA}) $$ $$ = \frac{1}{n_A^2}(V[\epsilon_{1A}]+V[\epsilon_{2A}]+2\cdot Cov[\epsilon_{1A},\epsilon_{2A}]) $$ $$ = \frac{1}{2^2}(\sigma^2+\sigma^2+2\cdot\sigma^2\rho) $$ $$ = \frac{1}{2}\sigma^2(1+\rho) $$ This is the formula I used. Now I have shown you how the first element in the covariance matrix is calculated, you should be able to figure out how to calculate the other diagonal elements from here.

  • 1
    $\begingroup$ You're goddamn right! Thank you! We have just abandoned the assumption of independence between errors... even if I wrote the initial assumptions I was still thinking that the variance-covariance matrix of the errors was $V[\varepsilon]=\sigma^2 I$ so I computed the variance above without even thinking about the covariance. Big mistake! $\endgroup$ – ChicagoCubs Sep 26 '15 at 13:44
  • 1
    $\begingroup$ Great! I know the feeling, sometimes I get into tunnel vision mode and it is hard to spot an error when it is obvious in front of you! Glad I could help :) $\endgroup$ – Gumeo Sep 26 '15 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.