I have used the TBATS model on my data and when I apply the forecast() function, it automatically forecasts two years in the future. I haven't specified any training set or testing set, so how do I know how to calculate RMSE, MASE, MAE and MAPE?

The data I'm dealing with is Uber travel times data from Jan 2016 to Jan 2020. I have daily data (sampling frequency = 1) for 18 cities and each city has a different sample size (they range from n = 1422 days to n = 1459 days).

Here is the example for data for London (n = 1458):


I have set the vector of travel times as an msts object, for it has multiple seasonality, which is used by the TBATS model.

Below is a graph with the fitted values and with the forecast values:

enter image description here

I mainly have 3 questions:

  1. How do I correctly calculate the KPIs for the forecast (RMSE, MASE, MAE, MSE)?
  2. What is the difference between finding out the accuracy of the red line to the black line, and the accuracy of the blue line?
  3. How do I know what testing data the forecast is being performed on? I only know that the forecast horizon is two years for tbats(). you can check forecast.tbats for assistance.

The values I'm getting for each are very low, here's my code:

data <- read.csv('C:/users/Datasets/Final Datasets/final_london.csv', TRUE, ",")
y <- msts(data$MeanTravelTimeSeconds, start=c(2016,1), seasonal.periods=c(7.009615384615385, 30.5, 91.3, 365.25))

fit <- tbats(y)
fc <- forecast(fit)

rmse <- sqrt(fit$variance)

res <- residuals(fit)
mae <- mean(abs(res))

pt <- (res)/y
mape <- mean(abs(pt))

# MSE (Mean Squared Error)
mse <- mean(res^2)

The performance was:

RMSE: 71.73491
MAE: 51.70627
MAPE: 0.02833748
MAE: 5145.897

Where the average travel time in London was around 1700 seconds.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.