Coefficient from Regressing the OLS Residual on X Say we have an OLS residual $\hat{ϵ}$ from regressing $y$ on $X$. If we were to regress $\hat{ϵ}$ on the same $X$, what would the OLS coefficient be?
If we rearrange the first regression equation for $\hat{ϵ}$, we get
$$
\hat{ϵ} =y−X\hat{β}
$$
but would the OLS coefficient simply be $\hat{β}$?
I suspect I am missing something.
 A: The residual vector from OLS estimation is uncorrelated with the explanatory vectors in the regression model, so the estimated coefficient vector from the regression you are proposing would be the zero vector.  To see this, first note that the residual vector can be written as:
$$\hat{\boldsymbol{\epsilon}}
= [\mathbf{I} - \mathbf{x} (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} ] \mathbf{y}.$$
Therefore, using OLS estimation with the design matrix $\mathbf{x}$ and response vector $\hat{\boldsymbol{\epsilon}}$ gives the estimated coefficient vector:
$$\begin{align}
\hat{{\beta}}_*
&= (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \hat{\boldsymbol{\epsilon}} \\[6pt]
&= (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} [\mathbf{I} - \mathbf{x} (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} ] \mathbf{y} \\[6pt]
&= [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} - (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \mathbf{x} (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} ] \mathbf{y} \\[6pt]
&= [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} - (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} ] \mathbf{y} \\[6pt]
&= \mathbf{0}. \\[6pt]
\end{align}$$
