# When “additive modeling” is better?

Given $y=f(x)+\epsilon$ where $x=(x_1,\dots,x_p)$, $f$ is highly non-linear and two different estimators:

1. $\hat{y}=\hat{M}(x)$

2. $\hat{y}=\hat M_1(x)+\hat M_2(x)$ where $M_1$ is a simple (biased) model and $M_2(x)=E[y-\hat M_1(x)|x]$.

Can you tell me when the second estimator (two-step modelling) will give better predictions ? If $f$ is highly non-linear, are $M_1$ and $M_2$ easier to estimate than $M$ ?

What about the bias/variance of the two estimators ? They have the same bias but what about the variance ?

To clarify, is the $\hat{M}$ in the first equation the same as $\hat{M}_1$ in the second? Or is it a joint estimation of $M_1$ and $M_2$? Or... –  jbowman Feb 26 '12 at 18:48
$M$ in the first equation can be a neural network with the best structure for the given data. $M_1$ is a biased model (for example, a non-optimal neural network for the given data) and $M_2$ models the remaining information not modeled by $M_1$. –  Souhaib Ben Taieb Feb 27 '12 at 7:35