Applying different time series models (ARIMA, HOLT-WINTER) on the basis of MAPE

Question

I have a time series object calc_visit_ts. I want to apply the best fit time series model based on the MAPE value for each model. The issue I face is that the MAPE value HOLT-WINTER multiplicative model cannot be calculated in the same way as the other models(as it gives me a different MAPE value when compared to summary(visit_model_Hw_M)).

#### AUTO-ARIMA
visit_model_Arima <- auto.arima(calc_visit_ts)
# summary(visit_model_Arima)

#### HOLT-WINTER ADDITIVE
visit_model_Hw_A <- hw(calc_visit_ts,h=monthly_prediction,seasonal = "additive")
# summary(visit_model_Hw_A)

#### HOLT-WINTER MULTIPLICATIVE
visit_model_Hw_M <- hw(calc_visit_ts,h=monthly_prediction,seasonal = "multiplicative")
# summary(visit_model_Hw_M)


#### Calculating MAPE on models for best suit
model_Mape<- c( MAPE_model(visit_model_Arima)
                ,MAPE_model(visit_model_Hw_A))
                #,MAPE_model(visit_model_Hw_M))  this is not accurate

model_Mape=na.omit(model_Mape)
token<-which(min(model_Mape)==model_Mape)

if(length(token)>0)
{
  if(token==1)
    {visit_model<-visit_model_Arima
  }else if(token==2)
    {visit_model<-visit_model_Hw_A
  }else if(token==3)
    {visit_model<-visit_model_Hw_M
  }else 
  {
    ##EXCEPTION HANDLING  
  }
}

summary(visit_model)

And here is the function I use to perform MAPE calculation on the models -

MAPE_model <- function(visit_model) {
 #CHECK FOR ZERO CONDIITION  if(visit_model$x!=0)
 mape = mean(abs(visit_model$residuals)/visit_model$x)
 return(mape)
}

Data for time series -

calc_visit_ts
     Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2012          35  53  65  60  64  49  63  55  59  66
2013  62  54  77  67  84  62  82  65  59  67  60  67
2014  73  75  55  76  93  96  89  76  88  65  83  82
2015  76  72  75  94  91  83  72  73  80  83  81  81
2016  97  91  90  80 101  98  

dput(calc_visit_ts)
structure(c(35, 53, 65, 60, 64, 49, 63, 55, 59, 66, 62, 54, 77, 
67, 84, 62, 82, 65, 59, 67, 60, 67, 73, 75, 55, 76, 93, 96, 89, 
76, 88, 65, 83, 82, 76, 72, 75, 94, 91, 83, 72, 73, 80, 83, 81, 
81, 97, 91, 90, 80, 101, 98), .Tsp = c(2012.16666666667, 2016.41666666667, 
12), class = "ts")

To show exactly what I mean -

Holt-Winter Additive Plot

Holt-Winter Multiplicative Plot

The issue is summary(visit_model_Hw_M) gives MAPE = 9.075097 whereas, MAPE_model(visit_model_Hw_M) gives 0.001273087 because the multiplicative model fits the curve(data points) therefore using visit_model_Hw_M$residuals isn't an appropriate way to calculate the MAPE(as the function tries to fit the curve).

Is there a way I can fetch the MAPE value for HOLT-WINTER multiplicative from the summary itself? OR a way to correctly estimate the MAPE value for the HOLT-WINTER multiplicative model?

Answer 1

After much deliberation over the data and trial and error I found that the MAPEs for ARIMA and Holt-Winter models are calculated differently -

MAPE_model <- function(visit_model,model_type) {
 if(model_type == "ARIMA")
  mape = mean(abs(visit_model$residuals)/visit_model$x)
 if(model_type == "HW")
  mape = mean(abs(visit_model$x - visit_model$fitted)/visit_model$x)
 else
  mape = -1  #Something
 return(mape)
}

While the notation visit_model$x - visit_model$fitted is canonically equivalent to visit_model$residuals in the ARIMA and Holt-Winter(additive) it is not the case in Holt-Winter(multiplicative) model. Hence, the distinction by model_type