# Goodness of forecast in R (Time Series)

I used two different methods to forecast a time series data.

1. The first one used is HoltWinters with Beta and Gamma as FALSE, since I don't see any trend or seasonality in the plot.

Below is the result from Box.test

Box.test(fore.holt.stat$residuals, type="Ljung-Box", lag=10) Box-Ljung test data: fore.holt.stat$residuals
X-squared = 10.691, df = 10, p-value = 0.3821


The p-value is 0.3821

1. I used auto.arima on the data and below is the result

Box.test(fore.arima$residuals, type="Ljung-Box", lag=10) Box-Ljung test data: fore.arima$residuals
X-squared = 14.724, df = 10, p-value = 0.1425


The p-value is 0.14

Question 1 :


Can I say that the first model is better since I have a higher p-vale?

Below are few other observations :

Model 1:

accuracy(fore.holt.stat)
ME     RMSE      MAE       MPE     MAPE     MASE       ACF1
Training set 424.9864 10275.55 7930.602 0.8782302 9.251837 0.766108 0.02142331


Model 2:

accuracy(fore.arima)
ME     RMSE     MAE        MPE     MAPE      MASE      ACF1
Training set 284.5242 7243.413 5371.42 -0.1874984 6.036736 0.5183941 0.0100049

Question 2 :


Which of the model is correct based on the accuracy function output?

In both the models, the p-value is high, but the mean of errors is not close to zero.

• Google 'forecast evaluation". It's a big area of forecasting. There are many criteria. – Aksakal Sep 28 '15 at 18:34

Which of the model is correct based on the accuracy function output?
• $p$-value has nothing to do with forecast accuracy, at least directly. Thus when interested in forecast accuracy, just look at the output of the accuracy function. That points to model 2 being preferred to model 1. However, I share your concern about the mean error. Ideally, ME indeed should be close to zero. If your observations are not measured in billions or so, the current ME looks poor. Maybe you are trying to fit the model to a data that clearly cannot be described by this kind of model?.. – Richard Hardy Sep 30 '15 at 6:07