1
$\begingroup$

For my master thesis I have implemented following forecasting models:

  • naive (just to check)
  • decomposition method
  • exponential smoothing (single/double/holt-winters)
  • SARIMA

Now I need to do the residuals check. However only residuals from SARIMA are white noise. Does it mean that all of the other models are bad? Or is white noise test not applicable for other methods?

To check the white-noise I performed Portmanteau test:

enter image description here

$\endgroup$
  • $\begingroup$ By white noise test, you mean Ljung-Box test or its variants? $\endgroup$ – mpiktas Nov 20 '14 at 13:25
  • $\begingroup$ Failure to reject doesn't imply the residuals are white noise. $\endgroup$ – Glen_b Nov 20 '14 at 14:01
  • $\begingroup$ Or perhaps your residuals are not white noise ! $\endgroup$ – aginensky Nov 20 '14 at 14:14
2
$\begingroup$

I would choose the model or method for which I cannot find evidence of any pattern or structure that could be incorporated in the model.

Answer to your second question: Regardless of the model and method, the presence of some structure in the residuals indicates that there is some room for improvement of the model. Roughly speaking, the residuals are the part that the model cannot explain. If we can find a pattern for the dynamics of the data that remains unexplained in the residuals, then we can improve the model.

Back to model selection: If we stick to the information provided by the Portmanteau test, I would choose the model for which, given a significance level chosen beforehand, the null of independence is not rejected.

However, it is recommended to gather more evidence and information. As noticed in one of the comments above, failing to reject the null does not imply that we must accept the null hypothesis; there may be patterns in the alternative hypothesis that are not detected or even considered by the test (e.g. heteroscedasticity).

In order to make a more knowledgeable decision, I suggest the following diagnostic tools (there are many other options which you have probably already considered): a plot of the autocorrelations and partial autocorrelations, a range-mean plot to check the homoscedasticity of residuals or a test for heteroscedasticy and a runs test to test the randomness of the sign of the residuals.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.