I have a daily observation of call volumes data starting from 28-01-2017 to 31-08-2018 a little over one and half year.On sundays calls volume are less and monday the highest showing weekly pattern. Plotting shows most of days in Nov month shows high call volumes above 2000. Values are also high in other days of different months but they are rare.

data_ts <- msts(data$Calls,seasonal.periods = c(7,365.25),start = c(2017,28))

Daily data

Data is divided into train test in 80:20 ratio and did dynamic harmonic regression on train data with fourier terms for weekly and annual seasonality. My residual analysis is pathetic and Mape on test data is 25. Increasing value of fourier terms (K in fourier function) not helping any way.

# creating xreg
xreg <- fourier(data_ts,K=c(1,1))
xreg_train <- xreg[1:448,]
xreg_test <- xreg[449:560,]

# fitting model
fit <- auto.arima(train,seasonal = FALSE,xreg = xreg_train)

Residual analysis

I think i need to work on the data first and then do forecasting. The boxplot of the series is shown as below.

enter image description here

My question is going forward how can i improve model performance to get better accuracy on test data. Do i need to pre-procees the series first and if yes what what should i look into.

EDIT: After doing little research i got some clue here Auto.arima with daily data: how to capture seasonality/periodicity? and created 6 weekly,11 monthly seasonal dummy variables,took 1 fourier terms and passed these additional information in xreg. Below is the xreg matrix

enter image description here

Now the residual analysis plot seems much better than before as shown below and test mape error come down to 20. But still serial correlation exists as seen in the acf plot. Ljung-Box test p value on residual is 0.00023 enter image description here

My objective is to catch those pattern in the residual and thereby possible getting less test mape error may be in single digit. Are there more possible ways to get there. Please suggest

  • $\begingroup$ There is still a lot of autocorrelation in your series, you need to neutralize that, $\endgroup$ – user2974951 Sep 27 '18 at 7:41
  • $\begingroup$ yeah...that is my query. How to do that? $\endgroup$ – joy_1379 Sep 27 '18 at 7:45
  • $\begingroup$ Hard to tell, you will have to play around with the arguments, I think there might be a need to add SMA and / or SAR processes. $\endgroup$ – user2974951 Sep 27 '18 at 7:47
  • $\begingroup$ autocorrelation can arise if there are untreated deterministic factors/features remaining in the residuals e.g. level shifts,local time trends,seasonal pulses ( read: day-of-the-week for example ) $\endgroup$ – IrishStat Sep 27 '18 at 10:38

Your question " going forward how can i improve model performance to get better accuracy on test data" . My answer "Build a better model that separates signal from noise" by using data-driven model identification tools.

Daily data often requires a model that contains both auto-projective and deterministic structure . See my answer to Transfer Function Equation from SPSS for some possible components. Simple method of forecasting number of guests given current and historical data presents a concrete example although the suggested approach is just not always simple but it is always thorough.

http://demand-planning.com/2010/03/18/can-forecasting-help-me-staff-a-specific-hewlett-packard-call-center-at-1030-am-on-a-friday/ discusses call center forecasting at an hourly level . The suggested methods can also be easily ( more easily ! ) implemented at a daily level.

  • $\begingroup$ @IrshStat - I took clue from your answer mentioned in the link in EDIT. Also to separate the noise dummy variables are used to capture extreme values. $\endgroup$ – joy_1379 Sep 27 '18 at 13:29
  • $\begingroup$ exactly ......... glad to be of help $\endgroup$ – IrishStat Sep 27 '18 at 13:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.