2
$\begingroup$

I want to know how accurate one ARIMA model is for estimating a second time series.

I've built the model:

mod1 <- auto.arima(x)
refit <- Arima(y, model=mod1)
> accuracy(refit)
               ME     RMSE MAE       MPE     MAPE MASE  ACF1
Training set -0.8 1.549193 1.2 -69.33333 77.33333  0.8 -0.05

Now I am not sure how to interpret these results. How good is the fit? Is there an accuracy or some measure like that?

$\endgroup$
1
$\begingroup$

Out of all the one simplest to understand is MAPE (Mean absolute percentage error). It considers actual values fed into model and fitted values from the model and calculates absolute difference between the two as a percentage of actual value and finally calculates mean of that.

For example if below are your actual data and results from ARIMA model

ActualData FittedValue AbsolutePercentageError
120        119.5       (abs(120-119)/120)*100 = 0.83%
128        126         (abs(128-126)/128)*100 = 1.56% 

MAPE = (0.83%+1.56%)/2 = 1.195%

Similarly you can do a quick google search to find out how meaning of other criterias. As per my experience MAPE is easiest one to explain to a layman, in case you want to explain model accuracy to a business user who is statistics illiterate. Also, you should forecast for a holdout sample for future and do the similar exercise to see how well it fits for future values Vs. the actuals.

Edit: This is an interesting discussion which accuracy metric to use in what scenario. Why use a certain measure of forecast error (e.g. MAD) as opposed to another (e.g. MSE)?

$\endgroup$
1
$\begingroup$

Here are a few points.

  1. Your ARIMA model is not "estimating a second time series", it is filtering it.

  2. The function accuracy gives you multiple measures of accuracy of the model fit: mean error (ME), root mean squared error (RMSE), mean absolute error (MAE), mean percentage error (MPE), mean absolute percentage error (MAPE), mean absolute scaled error (MASE) and the first-order autocorrelation coefficient (ACF1). It is up to you to decide, based on the accuracy measures, whether you consider this a good fit or not. For example, mean percentage error of nearly -70% does not look good to me in general, but that may depend on what your series are and how much predictability you may realistically expect.

  3. It is often a good idea to plot the original series and the fitted values, and also model residuals. You may occasionally learn more from the plot than from the few summarizing measures such as the ones given by the accuracy function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.