Projection vs fixed effects Suppose I have $n$ observation indexed by $i$ and that each observation is part of a group $g$. I want to compare two regressions.
First regression:
$$
Y_i=\beta X_i + \alpha F_{g(i)}+\varepsilon_i
$$
where $F$ is a set of group fixed effects.
Second regression:
$$
Y_i=\beta \hat{X}_i +\varepsilon_i
$$
where $\hat{X}$ is the residual from the regression
$$
X_i = \alpha F_{g(i)} + \varepsilon^0_i
$$
i.e. $\hat{X}_i = X_i - \hat{\alpha}F_{g(i)}$.
Are both regressions equivalent? I.e. are the point estimate, standard error, etc. of $\beta$ the same in both regressions?
It intuitively feel like both should be the same (the fixed effects absorb the variation due to the groups). But I would like a formal proof of this result. Many thanks!
 A: The two regression produce numerically equivalent estimates of $\beta$, but do not lead to the same predicted values of the outcome, nor do they share the same residuals. The standard errors are identical as well (Proof ommitted but sufficient to show the estimates are identical).
Proof (This follows closely from Davidson and McKinnon Ch. 2.4 on the FWL theorem):
I will use $P_Z = Z(Z^TZ)^{-1}Z^T$ to denote the projection matrix of Z and $M_Z = I - P_Z$ as the annihilator matrix of Z, that is the matrix which projects variables into the subspace orthogonal to the linear subspace spanned by $Z$, for arbitrary matrix $Z$
We can always decompose any outcome into the part in the subspace spanned by a set of variables, in this case the columns $Z = [X,F_{g}]$ and the part spanned by the orthogonal complement.
\begin{align}
Y &= P_ZY + M_ZY\\
&= X\hat{\beta} + F_{g}\hat{\alpha} + M_ZY
\end{align}
Where by definition $\hat{\beta}$ and $\hat{\alpha}$ are the vectors which minimize the distance between $Y$ and the linear subspace spanned by $[X,F_{g}]$ as is the case from regressing Y on X and $F_g$
Now
\begin{align}
Y &= X\hat{\beta} + F_{g}\hat{\alpha} + M_ZY\\
M_FY &= M_FX\hat{\beta} + M_FF_{g}\hat{\alpha} + M_FM_ZY\\
M_FY &=  M_FX\hat{\beta} + M_ZY\\
X^TM_FY &= X^TM_FX\hat{\beta} + X^TM_ZY\\
X^TM_FY &= X^TM_FX\hat{\beta}\\
(X^TM_FY)^{-1}(X^TM_FX) &= \hat{\beta}
\end{align}
Where line 2 follows from the fact that $M_FF = (I-P_F)F = F-P_XF = F- F = 0$, the zero vector and $M_FM_Z = M_Z$ since $X \subset Z = [X, F_{g}]$ (Intuitively, anything orthogonal to Z must already be orthogonal to F, so it the operator would map anything in that space to itself).
We have thus derived the estimator $\hat{\beta}$ from the multiple regression Y on X and $F_g$.
Now consider the two-step estimator you described. In my notation $\hat{X} = X - F_g\hat{\gamma} = X - F_g(F_g^TF_g)^{-1}F_g^TX = X - P_FX = M_FX$, where $\hat{\gamma} = (F_g^TF_g)^{-1}F_g^TX$ is the coefficient estimator from regression of $X$ on $F_g$ using the standard formula (i.e $(X^TX)^{-1}X^TY$, for regressing $Y$ on $X$.).
Now consider the regression of $Y$ on $M_FX$, this is a simple regression and we can again use the familiar OLS solution
\begin{align}
\hat{\beta^{2 stage}} &= ((M_FX)^TM_FX)^{-1}(M_FX)^TY\\
&= (X^TM_FM_FX)^{-1}X^TM_FY\\
&= (X^TM_FX)^{-1}X^TM_FY\\
&= \hat{\beta}
\end{align}
Where this follows from the symmetry and idempotency properties that orthogonal projection matrices enjoy. So we see that the coefficients are the same. However, the predicted values of Y are not. Let $\hat{y}$ be the predicted values from the first regression and $\hat{y^{2stage}}$ the predictions from the two stage precedure.
\begin{align}
\hat{y} &= X\hat{\beta} + F_g\hat{\alpha}\\
&= X(X^TM_FX)^{-1}X^TM_FY + F_g(F_g^TM_XF_g)^{-1}F_g^TM_XY
\end{align}
and
\begin{align}
\hat{y^{2stage}} &= M_FX\hat{\beta^{2stage}}\\
&= M_FX(X^TM_FX)^{-1}X^TM_FY 
\end{align}
These are not the same, similarly one can show the residuals are not the same in general. In Ch. 2.4 of Davidson and McKinnon they show that if your two stage regression involved not just projecting out linear combination of $F_g$ out of $X$, but also did the same for $Y$ and then regress the residual on each other, i.e regress $M_FY$ on $M_FX$, the coefficients for X will be the same as the original and the residuals will be numerically equivalent. The predicted values will still not be equal in general.
