0
$\begingroup$

I have a time series object calc_visit_ts. I want to apply the best fit time series model based on the MAPE value for each model. The issue I face is that the MAPE value HOLT-WINTER multiplicative model cannot be calculated in the same way as the other models(as it gives me a different MAPE value when compared to summary(visit_model_Hw_M)).

#### AUTO-ARIMA
visit_model_Arima <- auto.arima(calc_visit_ts)
# summary(visit_model_Arima)

#### HOLT-WINTER ADDITIVE
visit_model_Hw_A <- hw(calc_visit_ts,h=monthly_prediction,seasonal = "additive")
# summary(visit_model_Hw_A)

#### HOLT-WINTER MULTIPLICATIVE
visit_model_Hw_M <- hw(calc_visit_ts,h=monthly_prediction,seasonal = "multiplicative")
# summary(visit_model_Hw_M)


#### Calculating MAPE on models for best suit
model_Mape<- c( MAPE_model(visit_model_Arima)
                ,MAPE_model(visit_model_Hw_A))
                #,MAPE_model(visit_model_Hw_M))  this is not accurate

model_Mape=na.omit(model_Mape)
token<-which(min(model_Mape)==model_Mape)

if(length(token)>0)
{
  if(token==1)
    {visit_model<-visit_model_Arima
  }else if(token==2)
    {visit_model<-visit_model_Hw_A
  }else if(token==3)
    {visit_model<-visit_model_Hw_M
  }else 
  {
    ##EXCEPTION HANDLING  
  }
}

summary(visit_model)

And here is the function I use to perform MAPE calculation on the models -

MAPE_model <- function(visit_model) {
 #CHECK FOR ZERO CONDIITION  if(visit_model$x!=0)
 mape = mean(abs(visit_model$residuals)/visit_model$x)
 return(mape)
}

Data for time series -

calc_visit_ts
     Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2012          35  53  65  60  64  49  63  55  59  66
2013  62  54  77  67  84  62  82  65  59  67  60  67
2014  73  75  55  76  93  96  89  76  88  65  83  82
2015  76  72  75  94  91  83  72  73  80  83  81  81
2016  97  91  90  80 101  98  

dput(calc_visit_ts)
structure(c(35, 53, 65, 60, 64, 49, 63, 55, 59, 66, 62, 54, 77, 
67, 84, 62, 82, 65, 59, 67, 60, 67, 73, 75, 55, 76, 93, 96, 89, 
76, 88, 65, 83, 82, 76, 72, 75, 94, 91, 83, 72, 73, 80, 83, 81, 
81, 97, 91, 90, 80, 101, 98), .Tsp = c(2012.16666666667, 2016.41666666667, 
12), class = "ts")

To show exactly what I mean -

Holt-Winter Additive Plot

Holt-Winter Additive Plot

Holt-Winter Multiplicative Plot Holt-Winter Multiplicative Plot

The issue is summary(visit_model_Hw_M) gives MAPE = 9.075097 whereas, MAPE_model(visit_model_Hw_M) gives 0.001273087 because the multiplicative model fits the curve(data points) therefore using visit_model_Hw_M$residuals isn't an appropriate way to calculate the MAPE(as the function tries to fit the curve).

Is there a way I can fetch the MAPE value for HOLT-WINTER multiplicative from the summary itself? OR a way to correctly estimate the MAPE value for the HOLT-WINTER multiplicative model?

$\endgroup$
5
  • $\begingroup$ What exactly is the problem? You say ...cannot be calculated in the same way as the other models (as it gives me a different MAPE value when compared to summary(visit_model_Hw_M)). Why is that? $\endgroup$ Jul 9 '16 at 14:15
  • $\begingroup$ Basically, I cannot use HW(multiplicative) because it tries to fit the curve. Calculating the resulting MAPE from visit_model_Hw_M$residuals leads me to a unexpected value. $\endgroup$
    – Ic3fr0g
    Jul 9 '16 at 14:36
  • $\begingroup$ Don't all models fit curves? What you need is extract residuals and get their MAPE, ain't that the case? $\endgroup$ Jul 9 '16 at 14:45
  • $\begingroup$ I've edited the question. Sure models fit curves, but not HW(multiplicative) more so than others. I shall add a plot of a difference between the HW(A) and HW(M) models to give a better idea. $\endgroup$
    – Ic3fr0g
    Jul 9 '16 at 15:08
  • 2
    $\begingroup$ Well, it seems that the multiplicative model fits the curve perfectly... But perhaps you could compare MAPEs of out-of-sample predictions? That could be more fair than comparing in-sample fits; the latter will be prone to overfitting. $\endgroup$ Jul 9 '16 at 15:28
0
$\begingroup$

After much deliberation over the data and trial and error I found that the MAPEs for ARIMA and Holt-Winter models are calculated differently -

MAPE_model <- function(visit_model,model_type) {
 if(model_type == "ARIMA")
  mape = mean(abs(visit_model$residuals)/visit_model$x)
 if(model_type == "HW")
  mape = mean(abs(visit_model$x - visit_model$fitted)/visit_model$x)
 else
  mape = -1  #Something
 return(mape)
}

While the notation visit_model$x - visit_model$fitted is canonically equivalent to visit_model$residuals in the ARIMA and Holt-Winter(additive) it is not the case in Holt-Winter(multiplicative) model. Hence, the distinction by model_type

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.