I have a time series of 61 monthly observations and I would like to build an ARIMA forecasting model. To test my model, I separated my data into a training set of the 51 first observations, and a test set with the remaining 10.

Now my question is, when I am plotting the ACF and PACF to get an idea of the appropriate p and q parameters I should use in my model, should I plot the ACF and PACF only on my training set? Knowing that my ARIMA will be applied on the training set to forecast 10 future values, that I will compare to the observed ones in my test set.

  • $\begingroup$ Just from an intuitive point I would use 12 months (1 year) instead of 10 months as test set. Than you display an entire year with seasonalities. $\endgroup$ – Ferdi Nov 21 '17 at 20:46

Unfortunately, you have to validate your model using just your training set.

When it comes to time series, you also need to be really careful that your training set and your test set have similar seasonality and trends. In other words, if you have ten years of data, and each year has a 12-month cycle, be sure not to split you test/train set in the middle of a year.

  • $\begingroup$ Why would you recommend not splitting in the middle of a year? library(forecast); plot(forecast(auto.arima(AirPassengers[1:138]),h=18)) makes perfect sense. After all, years are cyclical, and when the year starts is just a convention. $\endgroup$ – Stephan Kolassa Nov 21 '17 at 20:15
  • $\begingroup$ You won't get a full prediction cycle if you split it. So, while your prediction may work well on at the beginning of the year, it may not work as well at the end. In general though, your training set and your testing set should be representative. $\endgroup$ – RussellB. Nov 21 '17 at 20:19
  • $\begingroup$ Well, the training set in my example is representative. And why wouldn't I get a full prediction cycle? My code snippet forecasts 18 months ahead. Finally, why should the forecast work more or less well in the near or long term depending on where I cut the holdout sample? $\endgroup$ – Stephan Kolassa Nov 21 '17 at 20:22
  • $\begingroup$ 18 months is probably sufficient. I was concerned about the 51/10 split in the question.... With time-series, it will make a pretty big difference where you split and if your training set doesn't contain a full cycle, you might run into problems. Try shifting your prediction from 18 to 24 or 18 to 12, see it for yourself. $\endgroup$ – RussellB. Nov 21 '17 at 20:33
  • $\begingroup$ Thank you for the hint, I might add 2 more observations to my training set, just in case $\endgroup$ – Notna Nov 21 '17 at 20:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.