Incremental learning for LOESS time series model I am currently working on some time series data, I know I can use
LOESS/ARIMA model. 
The  data is written to a vector whose length is 1000, which is a queue,
updating every 15 minutes,  
Thus the old data will pop out while the new data push in the vector.
I can rerun the whole model on a scheduler, e.g. retrain the model every 15
minutes, that is,  Use the whole 1000 value to train the LOESS model, However it is very inefficient, as every time only one
value is insert while another 999 vlaues still same as last time.
So how can I achieve better performance?
Many thanks
 A: Let me re-formulate this into something more familiar to me.  The ARIMA is an analog to PID approximation.  I is integral.  MA is P.  the AR can be expressed as difference equations which are the D term.  LOESS is an analog to least squares fitting (high-tech big brother really).  
So if I wanted to improve a second order model (PID) what could be done?  


*

*First, I could use a Kalman Filter to update the model with a single
piece of new information.

*I could also look at something called "gradient boosted trees". 
Using an analog of them, I would make a second ARIMA model whose
inputs are both the raw inputs fed to the first, augmented with the
errors of the first.

*I would consider looking at the PDF of the errors for multiple modes.
If I could cluster the errors then I might want to split models, or
use a Mixture model to separate the inputs into sub-models.  The
submodels might be better at handling the local phenomenology better
than a single large-scale model.


One of the questions that I have failed to ask is "what does performance mean?".  If we do not have a clearly stated measure of goodness then there is no way to tell if a candidate method "improves".  It seems like you want better modeling, shorter compute time, and more efficient use of information.  Having ephemeris about the actual data can also inform this.  If you are modeling wind, then you can know where to look for augmenting models, or find transformations for your data that are useful.
A: This is a different question depending on whether you are using a loess or an ARIMA model.  I will answer just the loess question for now, as I suspect there are little efficiencies possible in the ARIMA case other than perhaps having a good set of starting values.
A loess model works by fitting a weighted regression to different subsets of the data.  Only a proportion of the data is used for each fit.  So each time you refit the model having dropped off one data point at one end and added another at the opposite end, you technically only need to fit the local regressions that use the first and last point.  All the local regressions in-between will be the same.  Exactly how many of these un-impacted local regressions there are will depend on your smoothing parameter in the loess.
You could hack whatever package you are using to fit your model so that it can take most of the local regressions from a previous fit, and only fit those that are needed at the beginning and end of the data.
However, it would seem to me this was only worth doing if the cost in extra programming time was materially less than the cost in computer time of just fitting the model from scratch each 15 minutes.  With only 1000 data points surely it's not such a big thing to fit the model from scratch each time.
