I have daily data points of the number of sales, but I am not looking at historic data only. My system delivers a new data point every day and in the evening I want to predict the number of sales tomorrow. The sales are typically pretty constant, but they might increase or decrease from time to time. After a change they will be constant for another reasonable amount of time.
Furthermore there might be seasonality which increases the sales by a certain amount for the duration of the seasonal event. I cannot be sure of it, but there might be weekly, monthly or yearly (but no other than that) seasonality, a combination of those or no seasonality at all. (Public) holidays are not considered.
I am using R and was looking at arima and triple exponential smoothing (Holt-Winters) models to use in order to predict the number of sales for the next day. I have to predict the number of sales for tomorrow for around 1000 different data sets: that's why I can not look at all of them by hand, plot acf or pacf or fine tune the models manually. The 1000 data sets share all the characteristics described above.
The Holt-Winters models are working fine until the constant value changes. This somehow messes up the predicted data. The problem with arima models is the computation time which is much higher compared with Holt-Winters. For a data set with data points of two years it takes around 2 seconds. Since the system should work with data sets that hold data for at least 10 years and I have a thousand of those this might not be acceptable, but it might be possible with a reasonable amount of parallelization. I think I will not be able to use auto.arima since it takes simply too much time.
I have a few questions:
- Is it possible to choose the same arima model that seems to work fine for one or two of my data sets, that I tested manually, for all 1000 data sets if they share the same characteristics?
- If I look for yearly seasonality the model catches weekly and monthly seasonality as well. Does it do that in a worse/better way than pure weekly or monthly models? Should I combine different models for different seasonalities in my case?
- Which model would you recommend using: Is arima the right way? How would you determine the parameters for the model?
- Is there any guide or best practice I can follow for such a problem where the data set grows every day and cannot be investigated manually?
- In general, independent of my problem: Do I get more accurate results if I predict every day using a model for yearly seasonality combined to once every year with the same model?
- Related to 5.: How long does it take until a change in the above explained constant data affects the predictions for tomorrow in a yearly/monthly/weekly model?
If you have any other hints apart from direct answers I would be very pleased as well.
Thank you so much for your help.
Update: It seems that one reason for my problems to get accurate results is the huge difference of sales between non-seasonal and seasonal days. It might happen that the number of sales is around 1 on days which are not affected by any seasonal events and bigger than 1000 on seasonally affected days.