1
$\begingroup$

I'm using the TBATS model of the forecast package (version 8.0) with Google Analytics data, to forecast web trafic containing multiseasonal effects (msts). Before asking my questions, I'll explain the methodology I used.

Methodology

Data

I have two year of daily data that contains two columns like these :

date           sessions
2015-01-01     2667
2015-01-02     3542
2015-01-03     2383
2015-01-04     2772
2015-01-05     7797
2015-01-06     7599

I created a data.frame with the Google Anlytics API, that includes the data until the end of February 2017, with the intention of using it as a train set.

I created another data.frame with with the month of March 2017, with the intention of using it as a test set.

Model

I did a multiseasonal time series, to take into account the daily, monthly and annual seasonality :

y.msts <- msts(gadata$sessions,seasonal.periods=c(7,30.4, 365.25))

fit <- tbats(y.msts, use.box.cox=NULL, use.parallel=TRUE, num.cores = NULL, 
             use.trend=NULL, use.damped.trend=NULL, use.arma.errors=TRUE,                           
             model=NULL)

Results

Here is the time series components, residuals and forecast I obtained :

components <- tbats.components(fit)
plot(components)

checkresiduals(fit)

fc <- forecast(fit)
plot(fc)

enter image description here

With the accuracy function of the forecast package, I obtained the following results with the training set :

accuracy(fc)

             ME       RMSE     MAE       MPE      MAPE     MASE       ACF1
Training set 99.63043 1008.008 672.8444 -0.372223 10.10871 0.3125927 -0.06044894

Questions

  1. How to include a test set in the accuracy function? Is the accuracy function the best way to evaluate a forecasting model?

  2. What is the best way to create a test set without changing the date in the code every time (for example, by always taking the last month as a test set)?

  3. Any recommandation to improve the overall methodology from a statistical point of view?

$\endgroup$

closed as off-topic by Michael Chernick, mdewey, Nick Cox, gung, whuber Apr 4 '17 at 21:11

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Michael Chernick, mdewey, Nick Cox, gung, whuber
If this question can be reworded to fit the rules in the help center, please edit the question.

0
$\begingroup$
  1. Predict on the test set using forecast, before calculate the accuracy.

  2. Using the last date of training data, plus 30. You can try the lubridatepackage, it process the date and times very comfortable.

  3. First, try the seasonal test, whether there is multiple seasonal effects. Second, try different parameters for model or different models, compared the predictions on test set. After that, do the residual test, whether it is random.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.