# ARIMA: How to interpret MAPE?

I am using the forecast package in R to generate an ARIMA model for my data. I started with the auto.arima function for a try and got a ARIMA(1,1,2) model.

         ar1      ma1     ma2
0.7734  -1.0773  0.1191
s.e.  0.0709   0.0962  0.0824


But my question is not about the model itself but more about the validation of the model accuracy in general. Therefore I used "accuracy(fit)" and obtained the following output:

                    ME     RMSE      MAE      MPE     MAPE     MASE        ACF1
Training set -1.580214 163.8034 94.91732 -4.18724 13.61585 1.029359 0.002118006


I interpreted the MAPE like, "on average, the forecast if off by 14%", which sounds fine for me. But on the other side the MASE is greater than 1, which means the model is worse than a naive model. (?)

When I plot a forecast obviously the model is really bad:

fc<-forecast(fit, level=80, h=100)
plot(fc)


I am confused about the interpretation of the accuracy parameters I got. Did I miss some R command or how can I interpret the accuracy? Thanks!

• "When I plot a forecast obviously the model is really bad" - could you explain why the model is "obviously" "really bad"? Given your data, with lots of unexplained peaks, the forecast looks quite fine to me. – Stephan Kolassa Mar 7 '15 at 21:05
• well probably there is no better model, but what I meant is that the data is hard to predict and thus I was expecting a higher MAPE value than 14%. – MikeHuber Mar 8 '15 at 2:00

• Note that accuracy(fit) gives you accuracy measures on the in-sample fit. You can't see the in-sample fit on the plot. And note that in-sample fit accuracy is not a reliable guide to out-of-sample forecast accuracy. +1 to Richard's answer. – Stephan Kolassa Mar 7 '15 at 21:11
• Since ARIMA has only lagged (rather than contemporaneous) values on the right hand side, fitted values of an ARIMA model coincide with 1-step-ahead in-sample forecasts. Thus in-sample MAPE can be obtained from the data vs. fitted values from ARIMA model. For out-of-sample MAPE, use rolling windows. If your sample size is $T$, take $k<T$ to be your window length. Estimate your model on data points 1 through $k$, then 2 through $k+1$, ..., $T-k+1$ through $T$. Forecast 1 step ahead from each window. Collect the forecasts across windows, collect the corresponding realized values. Calculate MAPE. – Richard Hardy Mar 8 '15 at 8:50