2
$\begingroup$

I have 15-minutely data (96 values per day) over several years for around 340 entities (i.e. 340 data sets or long ts).

Now my task is to forecast a 4-hour window (i.e. 16 observations) for each day in the test set (the last 30% of each entity), for which I use ARIMA models.

In my opinion, it does not make sense to take the whole ts for each entity into account for forecasting the 4-hour window. Because of that, my approach is to take a much shorter period on which I create the Model, for example the last week before the 4-hour window (i.e. 96*7 = 672 Last values).

However, because auto.arima in R takes around 2 minutes to estimate the parameters of the model, I want to estimate the parameters for each entity globally, Meaning: If I have a very long time series (e.g. 105216 data points for an entity), which is stationary and seasonal with a frequency of 96, how would be the approach to determine the Parameters for an ARIMA models which fits best to each 672-data-point-sub-ts (or, more intuitive, each week)?

my initial idea was to loop through each week of the test data, create one model for each week, store the results and compare them for each entity. however, as stated above this takes a lot of time. Is there a better approach?

Improve this question
$\endgroup$
5
  • $\begingroup$ Due to the high frequency of your seasonality and the multiple seasonality you might be better of with "tbats" than with "(S)ARIMA" $\endgroup$
    – Ferdi
    Commented Oct 31, 2018 at 8:50
  • 1
    $\begingroup$ I want to benchmark (S)ARIMA compared to other models, thats why I'm interested in the parameters. Once I have solid parameters, a model with 672 data points is created in less then a second. So I'm looking for a scientific approach to determine those, as opposed to just "guessing" $\endgroup$
    – bk_
    Commented Oct 31, 2018 at 9:01
  • $\begingroup$ You might want to look at stats.stackexchange.com/questions/254466/… as it deals with exogenous factors like day-of-the-week , holidays etc while incorporating daily aggregates and arima memory $\endgroup$
    – IrishStat
    Commented Oct 31, 2018 at 9:58
  • $\begingroup$ thanks for the hint, unfortunately it does no help me identify (global) ARIMA parameters... $\endgroup$
    – bk_
    Commented Oct 31, 2018 at 10:40
  • $\begingroup$ Rob Hyndman has once answered a question here on how to fit a model for multiple time series where the parameter values are identical across the time series. His solution might not work for your dataset that is so large (over $10^5$ observations), but you might try looking the thread up nevertheless. $\endgroup$
    – Richard Hardy
    Commented Oct 31, 2018 at 14:40

1 Answer 1

Reset to default
1
$\begingroup$

I found an answer to the problem.

for each of the 340 entities, I estimated an initial model using auto.arima, based on the last few values. This model is used for the first forecast of the 16-observation forecast. For the next forecasting period (i.e. on the next day), I use my pre trained model, and pass it to the Arima function, in that way the parameters for each entity do not need to be re-estimated. It seems like the models perform pretty well. Furthermore, the Arima update is quite fast.

some pseudo code:

# day 1 fit <- auto.arima(train, seasonal = TRUE)

train consists of the last 4 days, since data is 15-minutely, 96 values per day. Hence, train is a ts object with frequency = 96.

# all following days fit <- Arima(train2, model = fit)

Where train2 are the data points in between.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.