# Applying different time series models (ARIMA, HOLT-WINTER) on the basis of MAPE

I have a time series object calc_visit_ts. I want to apply the best fit time series model based on the MAPE value for each model. The issue I face is that the MAPE value HOLT-WINTER multiplicative model cannot be calculated in the same way as the other models(as it gives me a different MAPE value when compared to summary(visit_model_Hw_M)).

#### AUTO-ARIMA
visit_model_Arima <- auto.arima(calc_visit_ts)
# summary(visit_model_Arima)

# summary(visit_model_Hw_A)

#### HOLT-WINTER MULTIPLICATIVE
visit_model_Hw_M <- hw(calc_visit_ts,h=monthly_prediction,seasonal = "multiplicative")
# summary(visit_model_Hw_M)

#### Calculating MAPE on models for best suit
model_Mape<- c( MAPE_model(visit_model_Arima)
,MAPE_model(visit_model_Hw_A))
#,MAPE_model(visit_model_Hw_M))  this is not accurate

model_Mape=na.omit(model_Mape)
token<-which(min(model_Mape)==model_Mape)

if(length(token)>0)
{
if(token==1)
{visit_model<-visit_model_Arima
}else if(token==2)
{visit_model<-visit_model_Hw_A
}else if(token==3)
{visit_model<-visit_model_Hw_M
}else
{
##EXCEPTION HANDLING
}
}

summary(visit_model)


And here is the function I use to perform MAPE calculation on the models -

MAPE_model <- function(visit_model) {
#CHECK FOR ZERO CONDIITION  if(visit_model$x!=0) mape = mean(abs(visit_model$residuals)/visit_model$x) return(mape) }  Data for time series - calc_visit_ts Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec 2012 35 53 65 60 64 49 63 55 59 66 2013 62 54 77 67 84 62 82 65 59 67 60 67 2014 73 75 55 76 93 96 89 76 88 65 83 82 2015 76 72 75 94 91 83 72 73 80 83 81 81 2016 97 91 90 80 101 98 dput(calc_visit_ts) structure(c(35, 53, 65, 60, 64, 49, 63, 55, 59, 66, 62, 54, 77, 67, 84, 62, 82, 65, 59, 67, 60, 67, 73, 75, 55, 76, 93, 96, 89, 76, 88, 65, 83, 82, 76, 72, 75, 94, 91, 83, 72, 73, 80, 83, 81, 81, 97, 91, 90, 80, 101, 98), .Tsp = c(2012.16666666667, 2016.41666666667, 12), class = "ts")  To show exactly what I mean - Holt-Winter Additive Plot Holt-Winter Multiplicative Plot The issue is summary(visit_model_Hw_M) gives MAPE = 9.075097 whereas, MAPE_model(visit_model_Hw_M) gives 0.001273087 because the multiplicative model fits the curve(data points) therefore using visit_model_Hw_M$residuals isn't an appropriate way to calculate the MAPE(as the function tries to fit the curve).

Is there a way I can fetch the MAPE value for HOLT-WINTER multiplicative from the summary itself? OR a way to correctly estimate the MAPE value for the HOLT-WINTER multiplicative model?

• What exactly is the problem? You say ...cannot be calculated in the same way as the other models (as it gives me a different MAPE value when compared to summary(visit_model_Hw_M)). Why is that? – Richard Hardy Jul 9 '16 at 14:15
• Basically, I cannot use HW(multiplicative) because it tries to fit the curve. Calculating the resulting MAPE from visit_model_Hw_M$residuals leads me to a unexpected value. – Ic3fr0g Jul 9 '16 at 14:36 • Don't all models fit curves? What you need is extract residuals and get their MAPE, ain't that the case? – Richard Hardy Jul 9 '16 at 14:45 • I've edited the question. Sure models fit curves, but not HW(multiplicative) more so than others. I shall add a plot of a difference between the HW(A) and HW(M) models to give a better idea. – Ic3fr0g Jul 9 '16 at 15:08 • Well, it seems that the multiplicative model fits the curve perfectly... But perhaps you could compare MAPEs of out-of-sample predictions? That could be more fair than comparing in-sample fits; the latter will be prone to overfitting. – Richard Hardy Jul 9 '16 at 15:28 ## 1 Answer After much deliberation over the data and trial and error I found that the MAPEs for ARIMA and Holt-Winter models are calculated differently - MAPE_model <- function(visit_model,model_type) { if(model_type == "ARIMA") mape = mean(abs(visit_model$residuals)/visit_model$x) if(model_type == "HW") mape = mean(abs(visit_model$x - visit_model$fitted)/visit_model$x)
else
mape = -1  #Something
return(mape)
}


While the notation visit_model$x - visit_model$fitted is canonically equivalent to visit_model\$residuals in the ARIMA and Holt-Winter(additive) it is not the case in Holt-Winter(multiplicative) model. Hence, the distinction by model_type