For predictors $X_1$ and $X_2$, and a response $Y$, two models are fit:
Model 1: Linear regression with $Y$ as response and $X_1$ and $X_2$ as predictors (only the linear terms).
Model 2: A linear regression model is fit with $Y$ as response and $X_1$ as a predictor. Using the residuals $R$ for this model, another model is fit for $R$ against $X_2$.
Apparently, Model 1 and Model 2 are equivalent (fitted values are the same) only when linear terms of $X_1$ and $X_2$ are involved, and $X_1$ is not correlated with $X_2$. How can this be mathematically proved?
Additionally, how does this concept relate to the boosting of regression trees, wherein the updated residuals are used as a response in successive regression trees?