I have an online learning problem where every second (say) I receive a new observation $(x_1,x_2,y)$. I'd like to fit the following models: $$ y = f(x_1) + f(x_2)$$ and maybe $$ y = f(x_1,x_2) $$
In an offline setting I would simply run the data through R npreg package or the gam function. Those methods however are offline methods that compute the model once. Whenever a new datum shows up I need to recompute the whole regression all over again.
This is very wasteful and I was hoping I could compute a simple non-parametric regression the same way I use recursive least squares filters for the parametric case (which predicts very poorly in this case).