I'm using the
TBATS model of the
forecast package (version 8.0) with Google Analytics data, to forecast web trafic containing multiseasonal effects (
msts). Before asking my questions, I'll explain the methodology I used.
I have two year of daily data that contains two columns like these :
date sessions 2015-01-01 2667 2015-01-02 3542 2015-01-03 2383 2015-01-04 2772 2015-01-05 7797 2015-01-06 7599
I created a
data.frame with the Google Anlytics API, that includes the data until the end of February 2017, with the intention of using it as a train set.
I created another
data.frame with with the month of March 2017, with the intention of using it as a test set.
I did a multiseasonal time series, to take into account the daily, monthly and annual seasonality :
y.msts <- msts(gadata$sessions,seasonal.periods=c(7,30.4, 365.25)) fit <- tbats(y.msts, use.box.cox=NULL, use.parallel=TRUE, num.cores = NULL, use.trend=NULL, use.damped.trend=NULL, use.arma.errors=TRUE, model=NULL)
Here is the time series components, residuals and forecast I obtained :
components <- tbats.components(fit) plot(components) checkresiduals(fit) fc <- forecast(fit) plot(fc)
accuracy function of the
forecast package, I obtained the following results with the training set :
accuracy(fc) ME RMSE MAE MPE MAPE MASE ACF1 Training set 99.63043 1008.008 672.8444 -0.372223 10.10871 0.3125927 -0.06044894
How to include a test set in the
accuracyfunction? Is the
accuracyfunction the best way to evaluate a forecasting model?
What is the best way to create a test set without changing the date in the code every time (for example, by always taking the last month as a test set)?
Any recommandation to improve the overall methodology from a statistical point of view?