My question

I have a linear regression that contains some regressors that vary only at a group level and some that vary at the individual level.

Slide 8 of this suggests that the coefficients on the group-level regressors are only affected by group-level variation in the individual-level regressors (i.e. variation within groups for the individual-level regressors doesn't matter for the coefficients on the group-level regressors). How do I prove this?

How far I've got

Call the group-level regressors $\mathbf{X_1}$ (a $N \times K_1$ matrix) and the individual-level regressors $\mathbf{X_2}$ (a $N \times K_2$ matrix).

Using the partitioned regression formula, I've been able to show that if there's no group-level variation in $\mathbf{X_2}$, then the coefficients on $\mathbf{X_1}$ do not depend on $\mathbf{X_2}$. To see this, note that the partitioned regression formula states: \begin{equation} \mathbf{b_1} = (\mathbf{X_1}'\mathbf{X_1})^{-1}\mathbf{X_1}'(\mathbf{y} - \mathbf{X_2}\mathbf{b_2}) \end{equation} Define $\mathbf{G}$ to be the $N \times G$ matrix of groups identifiers with the element $G_{ig} = 1$ if the $i$th observation is in group $g$ and 0 otherwise. Then we can write $\mathbf{X_1} = \mathbf{G}\mathbf{\overline{X}_1}$ where $\mathbf{\overline{X}_1}$ is the $G \times K_1$ matrix of group-level regressors with duplicate rows removed. Then by simple manipulation we have: \begin{equation} \mathbf{X_1} = \mathbf{G}(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_1} \end{equation} Note that in this $(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'$ is a matrix that finds group-level averages of the matrix that post-multiplies it. Thus, if there is no group-level variation in $\mathbf{X_2}$, we will have $(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_2} = \mathbf{0}$

The result that if there's no group-level variation in $\mathbf{X_2}$, then the coefficients on $\mathbf{X_1}$ do not depend on $\mathbf{X_2}$ is straightforward. We can write: \begin{align} \mathbf{X_1}'\mathbf{X_2} &= [\mathbf{G}(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_1}]'\mathbf{X_2} \\ &= \mathbf{X_1}'\mathbf{G}(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_2} \\ &= \mathbf{0} \end{align} where the last line follows because $(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_2} = \mathbf{0}$. Thus the formula for $\mathbf{b_1}$ becomes: \begin{equation} \mathbf{b_1} = (\mathbf{X_1}'\mathbf{X_1})^{-1}\mathbf{X_1}'\mathbf{y} \end{equation} and so does not depend on $\mathbf{X_2}$.

The outstanding problem

But here's the problem: if there is group-level variation in $\mathbf{X_2}$ I have only shown that $\mathbf{X_1}'\mathbf{X_2}$ only depends on group-level variation in $\mathbf{X_2}$. It seems to me that it is still possible for individual-level variation in $\mathbf{X_2}$ to matter through its effect on $\mathbf{b_2}$.

The full formula for $\mathbf{b_1}$ with $\mathbf{b_2}$ eliminated is: \begin{equation} \mathbf{b_1} = (\mathbf{X_1}'\mathbf{M_2}\mathbf{X_1})^{-1}\mathbf{X_1}'\mathbf{M_2}\mathbf{y} \end{equation} where $\mathbf{M_2} = \mathbf{I} - \mathbf{X_2}(\mathbf{X_2}'\mathbf{X_2})^{-1}\mathbf{X_2}'$. I can't see how I can substitute in $\mathbf{X_1} = \mathbf{G}(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_1}$ and rearrange to end up only with group-level averages of $\mathbf{X_2}$.

  • 1
    $\begingroup$ Be careful with your interpretation! In the model of slide 8, all regressors vary only at the group level. That's crucial. Note, two, that the second is a weighted OLS regression. For your more general statement to hold, I believe you must require all non-group-level regressors to be orthogonal to the group-level regressors (which is vacuously true in slide 8). $\endgroup$
    – whuber
    Sep 30, 2018 at 19:53
  • $\begingroup$ I am referring to the final bullet on slide 8 of the lecture I linked to, where the regression equation includes individual-level variation ($X_{ist}$) and the text says "The within state variation doesn't matter for identification but may reduce standard errors." $\endgroup$
    – jonsh
    Sep 30, 2018 at 20:36
  • $\begingroup$ I have a related question: if the Y variable only varies at the group level, but the X variable is at the individual level, it doesn't make sense to run a regression at the individual level, right? $\endgroup$
    – pietro
    Nov 19, 2018 at 8:42


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.