Variance-Covariance Matrix of the errors of a linear regression

Question

Suppose you have 8 observations ($i=1,...,8$) from three different states (A, B, C) and you also know that observations for $i=1,2$ are from state A, for $i=3,4,5$ are from state B and for $i=6,7,8$ are from state C. You are trying to estimate parameters with a linear regression model where $\varepsilon_i$ is the error term. The assumptions on this error term are that: $E[\varepsilon_i]=0$, $V[\varepsilon]=\sigma^2$ and:

$$Cov[\varepsilon_i, \varepsilon_j]=\begin{cases} \sigma^2 \rho & \text{ if observation i comes from the same state of observation j} \\ 0 & \text{otherwise} \end{cases}$$

Now you have that:

$$\overline{\varepsilon_h}=\frac{1}{n_h} \sum_{i \in h} \varepsilon_i$$

where $h=A,B,C$

I'm asked to compute the variance-covariance matrix of $\overline{\varepsilon}$ (notated $V[\overline{\varepsilon}]$) so I've started to compute variances and covariances of $\overline{\varepsilon}$ for $h=A, \ B, \ C$.

• $V[\overline{\varepsilon}_A]=...=\frac{\sigma^2}{n_A}$
• $V[\overline{\varepsilon}_B]=...=\frac{\sigma^2}{n_B}$
• $V[\overline{\varepsilon}_C]=...=\frac{\sigma^2}{n_C}$
• $Cov[\overline{\varepsilon}_A, \overline{\varepsilon}_B]= Cov\left [ \frac{1}{n_A} \sum_{i \in A} {\varepsilon}_i, \frac{1}{n_B} \sum_{j \in B} {\varepsilon}_j \right ]=\frac{1}{n_A} \frac{1}{n_B} Cov\left [ \sum_{i = 1}^{2} \varepsilon_i, \sum_{j = 3}^{5} \varepsilon_j \right ] = \frac{1}{n_A} \frac{1}{n_B} \sum_{i = 1}^{2} \sum_{j = 3}^{5} Cov[\varepsilon_i, \varepsilon_j]\underset{i\neq j}=0$
• $Cov[\overline{\varepsilon}_A, \overline{\varepsilon}_C]=...= 0$
• $Cov[\overline{\varepsilon}_B, \overline{\varepsilon}_C]=...= 0$

This leads me to the following variance-covariance matrix:

$$V[\overline{\varepsilon}]=\sigma^2\begin{bmatrix} \frac{1}{n_A} &0 &0 \\ 0 & \frac{1}{n_B} &0 \\ 0 & 0 & \frac{1}{n_C} \end{bmatrix}=\sigma^2 \begin{bmatrix} \frac{1}{2} &0 &0 \\ 0 & \frac{1}{3} &0 \\ 0 & 0 & \frac{1}{3} \end{bmatrix}$$

which apparently is not the right one. I think there is something wrong with the covariances computed above, but I can't see what. Can you please help me?

Answer 1

Your assumptions in the beginning are: $$ Cov[\epsilon_i,\epsilon_j] = \sigma^2\rho $$ For the variables that correspond to observations from the same state.

Now when you calculate variance for each of the three components in the end, you have to take that into account, i.e. $$ V[\overline{\epsilon}_A] = V[\frac{1}{n_A}\sum_{i=1}^2\epsilon_{iA}] $$ $$ = \frac{1}{n_A^2}\sum_{i=1}^2\sum_{j=1}^2Cov(\epsilon_{iA},\epsilon_{jA}) $$ $$ = \frac{1}{n_A^2}(V[\epsilon_{1A}]+V[\epsilon_{2A}]+2\cdot Cov[\epsilon_{1A},\epsilon_{2A}]) $$ $$ = \frac{1}{2^2}(\sigma^2+\sigma^2+2\cdot\sigma^2\rho) $$ $$ = \frac{1}{2}\sigma^2(1+\rho) $$ This is the formula I used. Now I have shown you how the first element in the covariance matrix is calculated, you should be able to figure out how to calculate the other diagonal elements from here.