# How to choose better model based on AICc or RMSE in certain situation?

Which attribute should consider as best fitted model AICc or RMSE in auto.arima? I am having a case of

best$aicc <- Inf for(i in 1:25){ r = fourier(timeSeries,K=i) ar <- auto.arima(anotherTimeseries, xreg= r, seasonal = false) if(ar$aicc < best\$aicc){
best = ar /* this part is reached only once for my data, the only first value is set as best and the aicc value keep on increasing*/
}else{
acc <- accuracy(ar)
acc1 <- accuracy(best)
/* check the RMSE accuracy rate and the set lowest RMSE value to best*/
}
forecast(best, xreg = (best i value from pervious value), h= 104)
}


Now, The doubt is whether I need to choose a best fitting model based on aicc value or RMSE value check (in the else part). Which approach will be proper?

AICC                RMSE
1642.857        acc- 233.6344
acc1 - 234.3495

1651.623        acc- 233.3246
acc1 - 234.3495

acc- 232.7801
1656.273        acc1- 234.3495


RMSE value decreases in every step but AICC value increases. Which one would be better arima model ? Thanks in advance for the suggestions

• The Reason for choosing lowest AIC is clear. As well as am gonna using the model to forecast. Thanks, Richard. Commented Apr 25, 2017 at 23:24

forecast::accuracy will give you in-sample accuracy measures which are useless for model selection. By construction, a model with some additional Fourier terms will beat a model without them in sample. But we are normally interested in generalization performance, i.e. out of sample.
On the other hand, AICc estimates $$-2n\ \times$$ expected log-likelihood out of sample (as explained in this answer and can be found in Hastie et al. "The Elements of Statistical Learning") and is a sound criterion for model choice, especially if the goal is forecasting. You should pick the model with the lowest AICc.