# Residualizing in OLS

Consider a linear regression

$$y_i = A_i' \theta + B_i' \psi + \epsilon_i$$

There's a "trick" to find the parameters $$\theta$$ and $$\psi$$ in a stepwise fashion.

First minimizing squared error wrt. $$\theta$$: $$\hat{\theta}(\psi) = (A^t A)^{-1} A^t (y - B\psi)$$

Plugging back in, the residuals are $$\epsilon = ( I - A(A^t A)^{-1} A^t ) (y - B \psi)$$ so the idea is that to find $$\hat{\psi}$$ by residualizing both $$y$$ and $$B$$ with respect to $$A$$.

Let $$y_{\perp A} = (I - A(A^t A)^{-1} A^t) y$$ and $$B_{\perp A} = (I - A(A^t A)^{-1} A^t) B$$. Then you just do $$\hat{\psi} = (B_{\perp A}^t B_{\perp A})^{-1} B_{\perp A}^t y_{\perp A}$$

My questions are:

• What is this called? I've tried searching for "stepwise regression", but that's definitely not the right term
• Is it possible to extend this argument to a case when the dependence on $$\psi$$ is non-linear? I'm currently fitting the non-linear model to the residuals of the linear one, but I'm not sure if it's a valid thing to do

Many thanks