# Incremental learning for LOESS time series model

I am currently working on some time series data, I know I can use LOESS/ARIMA model.

The data is written to a vector whose length is 1000, which is a queue, updating every 15 minutes,

Thus the old data will pop out while the new data push in the vector.

I can rerun the whole model on a scheduler, e.g. retrain the model every 15 minutes, that is, Use the whole 1000 value to train the LOESS model, However it is very inefficient, as every time only one value is insert while another 999 vlaues still same as last time.

So how can I achieve better performance?

Many thanks

-
What is a "LOESS/ARIMA model"? Do you meant a nonlinear time trend with ARIMA errors where the nonlinear trend is estimated using LOESS? –  Rob Hyndman Dec 20 '10 at 22:33
sorry I mean LOESS or ARIMA model. e.g I use LOESS to find the the residuals as: residuals(loess(x ~ time)). because the data x is vector with 1000 values, which updates every 15 minutes. How can I efficiently get the residuals, but not rerun the whole datset as input everytime? as only 1 value update everytime, the other 999 values is still same as last time. –  zhang Dec 21 '10 at 13:16
one possible method, maybe use the first 1000 values to predict the next 1000 values(although LOESS only support predict 4 values), then calculate the residual as the difference between actual value and corresponding predict value.Then retrain the model every 1000 values However, this is not the original LOESS model I want at all :-( –  zhang Dec 22 '10 at 16:59

This is a different question depending on whether you are using a loess or an ARIMA model. I will answer just the loess question for now, as I suspect there are little efficiencies possible in the ARIMA case other than perhaps having a good set of starting values.

A loess model works by fitting a weighted regression to different subsets of the data. Only a proportion of the data is used for each fit. So each time you refit the model having dropped off one data point at one end and added another at the opposite end, you technically only need to fit the local regressions that use the first and last point. All the local regressions in-between will be the same. Exactly how many of these un-impacted local regressions there are will depend on your smoothing parameter in the loess.

You could hack whatever package you are using to fit your model so that it can take most of the local regressions from a previous fit, and only fit those that are needed at the beginning and end of the data.

However, it would seem to me this was only worth doing if the cost in extra programming time was materially less than the cost in computer time of just fitting the model from scratch each 15 minutes. With only 1000 data points surely it's not such a big thing to fit the model from scratch each time.

-