# How to use auto.arima when attaining different results with a manual model selection?

I have daily sales data for over 2 years. I am having several questions of how to manually define model order and how to do it using auto_arima.

Manual

The first step I do it to see if my data is stationary, if it is then d = 0; otherwise I do a check to see what my d should be. In the case of my data, using the Dicky Fuller test I can accept the alternative hypothesis that my data is stationary on the 95% significance level (p < 0.05). Now here arises my first question.

I want to be able to predict 28 days ahead, and want to be able to see how my model scores using a test set. So I hold out (28 days) for this.

Now when I do the stationarity check, do I have to do it on the whole dataset, or only on the training dataset?

Next up I plot my ACF and PACF plots and see spikes every 7 lags (so 7, 14, 21, ...). So I assume my model orders for AR and MA to be 7. This concludes to an ARMA model (7, 0, 7)?

Estimate selection

Now when I use the auto_arima function. First of again, do I use the whole dataset to find the model order, or only the training dataset? If I use the whole dataset, I assume I only fit the model on the training dataset?

The thing is, whatever I do (whole dataset, or only training dataset), I get very different model order results, ARMA(6,0,1). Even though I specify max_p and max_q to be 7. When comparing these models on the test set, the ARMA(7, 0, 7) performs way better (the selection happends based on AIC, and I assume that that is the reason since it penalizes having more features?).

Concluding question

The question thus is, if I want to split up my dataset in training and test dataset, do I only use the training dataset for the checks (Dicky Fuller & auto_arima)? And also why does my manual + intuitively model not occurr in the estimate selection?

• Why do you use underscore in place of a dot? Or are these not R functions and their arguments you are referring to? – Richard Hardy Jun 12 '20 at 20:28
• It's the python counterpart, whic I think is auto_arima – Tibo Geysen Jun 12 '20 at 20:30
• OK. I just thought if some people follow questions about auto.arima (the popularity of which absolutely dominates those on auto_arima here at Cross Validated), they may miss the ones about auto_arima just because of the unfortunate punctuation. – Richard Hardy Jun 12 '20 at 20:35
• I'll try and convert my code to the R counterpart and see what results I get, but I guess the problem will stay the same – Tibo Geysen Jun 13 '20 at 10:18