I have used the TBATS model on my data and when I apply the forecast() function, it automatically forecasts two years in the future. I haven't specified any training set or testing set, so how do I know how to calculate RMSE, MASE, MAE and MAPE?
The data I'm dealing with is Uber travel times data from Jan 2016 to Jan 2020. I have daily data (sampling frequency = 1) for 18 cities and each city has a different sample size (they range from n = 1422 days to n = 1459 days).
Here is the example for data for London (n = 1458):
I have set the vector of travel times as an msts
object, for it has multiple seasonality, which is used by the TBATS model.
Below is a graph with the fitted values and with the forecast values:
I mainly have 3 questions:
- How do I correctly calculate the KPIs for the forecast (RMSE, MASE, MAE, MSE)?
- What is the difference between finding out the accuracy of the red line to the black line, and the accuracy of the blue line?
- How do I know what testing data the forecast is being performed on? I only know that the forecast horizon is two years for
tbats()
. you can checkforecast.tbats
for assistance.
The values I'm getting for each are very low, here's my code:
data <- read.csv('C:/users/Datasets/Final Datasets/final_london.csv', TRUE, ",")
y <- msts(data$MeanTravelTimeSeconds, start=c(2016,1), seasonal.periods=c(7.009615384615385, 30.5, 91.3, 365.25))
fit <- tbats(y)
fc <- forecast(fit)
# RMSE
rmse <- sqrt(fit$variance)
# MAE
res <- residuals(fit)
mae <- mean(abs(res))
# MAPE
pt <- (res)/y
mape <- mean(abs(pt))
# MSE (Mean Squared Error)
mse <- mean(res^2)
The performance was:
RMSE: 71.73491
MAE: 51.70627
MAPE: 0.02833748
MAE: 5145.897
Where the average travel time in London was around 1700 seconds.