# My question

I have a linear regression that contains some regressors that vary only at a group level and some that vary at the individual level.

Slide 8 of this suggests that the coefficients on the group-level regressors are only affected by group-level variation in the individual-level regressors (i.e. variation within groups for the individual-level regressors doesn't matter for the coefficients on the group-level regressors). How do I prove this?

# How far I've got

Call the group-level regressors $$\mathbf{X_1}$$ (a $$N \times K_1$$ matrix) and the individual-level regressors $$\mathbf{X_2}$$ (a $$N \times K_2$$ matrix).

Using the partitioned regression formula, I've been able to show that if there's no group-level variation in $$\mathbf{X_2}$$, then the coefficients on $$\mathbf{X_1}$$ do not depend on $$\mathbf{X_2}$$. To see this, note that the partitioned regression formula states: $$\begin{equation} \mathbf{b_1} = (\mathbf{X_1}'\mathbf{X_1})^{-1}\mathbf{X_1}'(\mathbf{y} - \mathbf{X_2}\mathbf{b_2}) \end{equation}$$ Define $$\mathbf{G}$$ to be the $$N \times G$$ matrix of groups identifiers with the element $$G_{ig} = 1$$ if the $$i$$th observation is in group $$g$$ and 0 otherwise. Then we can write $$\mathbf{X_1} = \mathbf{G}\mathbf{\overline{X}_1}$$ where $$\mathbf{\overline{X}_1}$$ is the $$G \times K_1$$ matrix of group-level regressors with duplicate rows removed. Then by simple manipulation we have: $$\begin{equation} \mathbf{X_1} = \mathbf{G}(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_1} \end{equation}$$ Note that in this $$(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'$$ is a matrix that finds group-level averages of the matrix that post-multiplies it. Thus, if there is no group-level variation in $$\mathbf{X_2}$$, we will have $$(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_2} = \mathbf{0}$$

The result that if there's no group-level variation in $$\mathbf{X_2}$$, then the coefficients on $$\mathbf{X_1}$$ do not depend on $$\mathbf{X_2}$$ is straightforward. We can write: \begin{align} \mathbf{X_1}'\mathbf{X_2} &= [\mathbf{G}(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_1}]'\mathbf{X_2} \\ &= \mathbf{X_1}'\mathbf{G}(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_2} \\ &= \mathbf{0} \end{align} where the last line follows because $$(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_2} = \mathbf{0}$$. Thus the formula for $$\mathbf{b_1}$$ becomes: $$\begin{equation} \mathbf{b_1} = (\mathbf{X_1}'\mathbf{X_1})^{-1}\mathbf{X_1}'\mathbf{y} \end{equation}$$ and so does not depend on $$\mathbf{X_2}$$.

# The outstanding problem

But here's the problem: if there is group-level variation in $$\mathbf{X_2}$$ I have only shown that $$\mathbf{X_1}'\mathbf{X_2}$$ only depends on group-level variation in $$\mathbf{X_2}$$. It seems to me that it is still possible for individual-level variation in $$\mathbf{X_2}$$ to matter through its effect on $$\mathbf{b_2}$$.

The full formula for $$\mathbf{b_1}$$ with $$\mathbf{b_2}$$ eliminated is: $$\begin{equation} \mathbf{b_1} = (\mathbf{X_1}'\mathbf{M_2}\mathbf{X_1})^{-1}\mathbf{X_1}'\mathbf{M_2}\mathbf{y} \end{equation}$$ where $$\mathbf{M_2} = \mathbf{I} - \mathbf{X_2}(\mathbf{X_2}'\mathbf{X_2})^{-1}\mathbf{X_2}'$$. I can't see how I can substitute in $$\mathbf{X_1} = \mathbf{G}(\mathbf{G}'\mathbf{G})^{-1}\mathbf{G}'\mathbf{X_1}$$ and rearrange to end up only with group-level averages of $$\mathbf{X_2}$$.

• Be careful with your interpretation! In the model of slide 8, all regressors vary only at the group level. That's crucial. Note, two, that the second is a weighted OLS regression. For your more general statement to hold, I believe you must require all non-group-level regressors to be orthogonal to the group-level regressors (which is vacuously true in slide 8). – whuber Sep 30 '18 at 19:53
• I am referring to the final bullet on slide 8 of the lecture I linked to, where the regression equation includes individual-level variation ($X_{ist}$) and the text says "The within state variation doesn't matter for identification but may reduce standard errors." – jonsh Sep 30 '18 at 20:36
• I have a related question: if the Y variable only varies at the group level, but the X variable is at the individual level, it doesn't make sense to run a regression at the individual level, right? – pietro Nov 19 '18 at 8:42