Given $y=f(x)+\epsilon$ where $x=(x_1,\dots,x_p)$, $f$ is highly non-linear and two different estimators:
$\hat{y}=\hat{M}(x)$
$\hat{y}=\hat M_1(x)+\hat M_2(x)$ where $M_1$ is a simple (biased) model and $M_2(x)=E[y-\hat M_1(x)|x]$.
Can you tell me when the second estimator (two-step modelling) will give better predictions ? If $f$ is highly non-linear, are $M_1$ and $M_2$ easier to estimate than $M$ ?
What about the bias/variance of the two estimators ? They have the same bias but what about the variance ?
Is there a link with additive modeling ?
