7
$\begingroup$

I would like to have the best ARIMA model prediction that has the lowest MAPE or lowest AIC/BIC. For example, I would want to change the Arima order automatically with loop or some other way and want to test with all possible combinations like below

c(1,0,0)
c(1,1,0)
.
.
c(x,y,z)

Below is the reproducible example code but I do not know how to go with multiple order execution and comparison of MAPE/AIC/BIC.

set.seed(1)
tsdata <- ts(rnorm(50), start = c(1980,1), frequency = 12)
myts <- tsdata

fit <- Arima(myts,order=c(2,1,0))
forecast(fit, 3)
plot(forecast(fit, 3))
fit
accuracy(fit)

Is it possible to save all the Accuracy measures (MAPE, AIC, BIC) in a data frame or in a list then select the best order to execute a Arima model? I tested with auto.arima in my real data but it did not give me the best order. Thanks in advance for your help !

$\endgroup$
5
  • $\begingroup$ Generally question about how to get the "best" model require more statistical knowledge and correct use of terminology. Perhaps you should first get some statistical consultation, either at you business or school or CrossValitdated.com. $\endgroup$
    – DWin
    Dec 16, 2014 at 17:38
  • $\begingroup$ Though, this is about statistics I feel R programmers can do this. Because I hope this can be dealt with loops or other way. We need to use different arima order combination and store the results in a list or data frame and then compare and select the best one. Does this make sense? $\endgroup$
    – Learner
    Dec 17, 2014 at 2:29
  • 2
    $\begingroup$ MAPE is a very bad performance measure for data close to zero. Consider using MASE, or at least MAE. $\endgroup$
    – mpiktas
    Jan 6, 2015 at 14:21
  • $\begingroup$ can you provide little more detail on what you tried, that seems to be important to answer your question. in auto.arima you have an option to select the best model from all possible candidate models by specifying stepwise = FALSE which is not available by default. In addition, there could be outliers inyour data, if not properly treated no matter what order you choose, the accuracy could not be improved. $\endgroup$
    – forecaster
    Jan 6, 2015 at 20:02
  • 2
    $\begingroup$ As an example, using AIC as a criterion, auto.arima in this example using the following code x <- auto.arima(AirPassengers,trace=TRUE,ic="aic",stepwise=F) evalauted 96!!! models and chose the best model based on the AIC criterion. $\endgroup$
    – forecaster
    Jan 6, 2015 at 20:06

1 Answer 1

3
$\begingroup$

This was the first that came to mind, and is just an example but it is slow and it should be done once per integration order you wish to consider. Looping through the 10 first AR and MA orders (again only example), saving the MAPE accuracy measures in a matrix "x". Then identifying the smallest value in the matrix afterwards. Is it something like this you had in mind?

    set.seed(1)
tsdata <- ts(rnorm(50), start = c(1980,1), frequency = 12)
myts <- tsdata

x <- matrix(data = NA, nrow=10, ncol=10)
for(i in 0:9){
  for(j in 0:9){
    fit <- arima(myts, order=c(i,1,j))
    acc <- accuracy(fit)
    x[i+1,j+1] <- acc[[5]] # Number 5 indicates the position of MAPE in the accuracy list
    print(i);print(j)
  }

}

which(x==min(x), arr.ind=T)
row col
[1,]   3   1

edit: deleted unnecessary code.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.