To find the best neural network auto regressive model for forecasting future values of a series with 180 month of observation I plotted ACF and PACF of series as follows Using Forecast package in R.

enter image description here

From PACF plot I decided to have 2 non- seasonal (p=2) and 3 seasonal (P=3) terms in my model using nnatar(p,P) function. Fitted values versus observed (R2=0.56) and also forecasted values for next 12 month with CI 80% and 95% are presented in following figure. enter image description here

I intentionally without any logic increased the non-seasonal terms from 2 to 10 only to compare results with former model. The produced plot in this case is as follows which show the fitted values very better fitted on observed values (R2=0.99) and confidence interval for forecasted values is also pretty narrow. enter image description here

My question is if selection of the number of the model predictors (seasonal and non-seasonal) should be based on PACF plot, how we can judge to select the best model while increasing the input lags will increase the model R2. I am worry about model over-fitting in second model. I am sure the behavior of my series could not be completely described by its lags. Thus certainly there may be some other exogenous variable that may have an influential effect on this series but in this model I only want to consider series lags effect. Thus I think the first model may be better selection because it described only 56% of variance but the second 99% which is not reliable! However, I am not sure about this my selection…. Please help me know what cases is better.

  • $\begingroup$ how did you calculate R2 by using nnetar function? $\endgroup$ – oay Mar 10 at 12:35

Before fitting a nnetar() model you might try the following:

  1. Is the series homoscedastic? If unsure, try with option lambda="auto" to automatically select the transformation is required to stabilise the variance.

  2. Fit auto.arima() (with and without lambda inside this function) to detect both seasonal and non-seasonal AR. Then use that in the nnetar() model.

  3. Both nnetar() and auto.arima() has options to include exogenous series through xreg option. Might be helpful to fit models with and without this option to evaluate both point forecasts and prediction intervals.

  4. Instead of fitting one single model, you might also think of fitting several models on sliding window to figure out whether one particular model is better in terms of fitting (AIC/BIC) and in terms of forecasting (MSE/MAPE etc.)

Hope this helps!

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.