1
$\begingroup$

I am running a ARIMA model on my data. I have weekly data from Jan 2021. When I run 12 weeks forecast, the ARIMA gives the best parameter values (0,0,0) indicating that the data is white noise. But when I use the same data and run a 8 weeks forecast, the best parameter values are (4,0,4), which means data isn't white noise.

Could someone please let me know how is forecast duration making the data white noise?

So here's how I am running my forecasting:

Data runs from Jan 2021 to July 2024.

Training data: Jan 1, 2021 to Feb 11, 2024.

Test Data: Feb 18, 2024 to May 6, 2024.

I loop over large sets of p,d,q values using:

params = {
          'p' : [0,1,2,3,4,5],
          'd' : [0,1,2,3,4,5],
          'q' : [0,1,2,3,4,5]
}

for vals in product(*params.values()):
    comb = dict(zip(params.vals)

then I pass comb into my function that performs ARIMA.

After the above step, I select the best p,d,q values based on lowest RMSE scores based on test vs forecast values. and pass them onto the out to the next 12 weeks forecast:

Now training data: Jan 1, 2021 to May 13, 2024 test data: May 13, 2024 to July 29, 2024

Improve this question
$\endgroup$
8
  • $\begingroup$ What exactly do you mean by When I run $h$ weeks forecast, the ARIMA gives the best parameter values (p,d,q)? $\endgroup$
    – Richard Hardy
    Commented Aug 14 at 15:54
  • $\begingroup$ @RichardHardy I am iterating over parameter space using itertoos product. It will go through different combinations of p,d,q values and the combination that gives lowest RMSE, I referred as "best parameter values" $\endgroup$
    – Karthik S
    Commented Aug 16 at 15:24
  • 1
    $\begingroup$ By RMSE, do you mean out-of-sample root mean square forecast error? Generally, ARIMA is not great at long-horizon forecasts, so no wonder if ARIMA(0,0,0) is the best you can get for 12 steps ahead. I am not familiar with itertools, but if you are doing grid search over a large number of possible combinations of p, d, q, then you might as well be overfitting on the test set, so the model choice is not necessarily sensible. $\endgroup$
    – Richard Hardy
    Commented Aug 16 at 16:05
  • $\begingroup$ @RichardHardy I've updated my question with the steps I take to perform the forecast. Thanks for the info. So given my scenario, could you please let me know how to go about forecasting please? Also, when I run 8 weeks forecast instead of 12, the data for states no longer is white noise, i.e. the best p,d,q parameters are not (0,0,0), Could you please let me know why that'could be the case $\endgroup$
    – Karthik S
    Commented Aug 19 at 10:09
  • 1
    $\begingroup$ I have included my suggestions in my answer. I do not have much more to say beyond that. Regarding model choice and the use of training, validation and test sets, I think your current strategy is reasonable. You could use rolling windows instead of a single split into the three subsets to use your data a little bit more efficiently, but I do not expect that to be a game changer. Just keep your model selection/tuning separated from model evaluation (as you have done so far) to avoid deceiving yourself :) $\endgroup$
    – Richard Hardy
    Commented Aug 19 at 13:26

1 Answer 1

Reset to default
2
$\begingroup$

In my experience, ARIMA with nonzero $p$ and/or $q$ values is not great for long-horizon forecasts, so no wonder if ARIMA(0,0,0) is the best you can get for 12 steps ahead.

I am not familiar with itertools, but if you are doing grid search over a large number of possible combinations of $p$, $d$, $q$, then you might well be overfitting on the validation set, so the chosen model is not necessarily a good one. For that reason, I think I would trust a model chosen by auto.arima using its default settings more than one chosen by your extensive grid search. (auto.arima tries fewer combinations of $p$, $d$, $q$ when using the default settings, so it has a lower chance of overfitting.) If you do not like auto.arima, I would at least suggest that you could save 2/3 of your computational time and avoid the possibility of choosing a very poor (but lucky on this particular validation sample) model model by dropping $d=2,3,4,5$.

If I understand you correctly, you are tuning $p$, $d$, $q$ on a validation set and evaluating the chosen model on a separate test set. If so, at least you get a fair evaluation of the chosen model's performance, even if that model should be a poor choice due to overfitting.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.