Comparison of horizon forecasted values for two models (ARIMA and NAR)?

Question

I fitted ARIMA and NAR models to 10 different univariate time series. These time series are water characteristics at a water treatment plant (CO$_2$, TDS, TH, pH, ...).

After fitting suitable models I forecasted seven lags ahead with models for each series. I have seven observed at the end of each series that should be plotted with the corresponding seven forecasted. After plotting I found that the forecasted values by NAR model are visually near the observed but the values by ARIMA were almost constant across observed values.

For more comparison I decided to calculate MSE for the forecasted values for two models. My expectation was that MSE for NAR should be smaller than ARIMA but this was completely different. For example, for one of my series forecasted values by NAR were visually close in comparison with those from ARIMA but its MSE was more than for ARIMA.

Why? Please let me how can I compare them statistically.

Answer 1

You might want to read my answer to auto.arima and prediction as it discusses the difference between fitted errors which actually use real previous values and forecast errors which are self-propagating where forecasts are used as pseudo-actuals to create the next forecast.