I have used the TBATS model on my data and when I apply the forecast() function, it automatically forecasts two years in the future. I haven't specified any training set or testing set, so how do I know how to calculate RMSE, MASE, MAE and MAPE?
The data I'm dealing with is Uber travel times data from Jan 2016 to Jan 2020. I have daily data (sampling frequency = 1) for 18 cities and each city has a different sample size (they range from n = 1422 days to n = 1459 days).
Here is the example for data for London (n = 1458):
I have set the vector of travel times as an
msts object, for it has multiple seasonality, which is used by the TBATS model.
Below is a graph with the fitted values and with the forecast values:
I mainly have 3 questions:
- How do I correctly calculate the KPIs for the forecast (RMSE, MASE, MAE, MSE)?
- What is the difference between finding out the accuracy of the red line to the black line, and the accuracy of the blue line?
- How do I know what testing data the forecast is being performed on? I only know that the forecast horizon is two years for
tbats(). you can check
The values I'm getting for each are very low, here's my code:
data <- read.csv('C:/users/Datasets/Final Datasets/final_london.csv', TRUE, ",") y <- msts(data$MeanTravelTimeSeconds, start=c(2016,1), seasonal.periods=c(7.009615384615385, 30.5, 91.3, 365.25)) fit <- tbats(y) fc <- forecast(fit) # RMSE rmse <- sqrt(fit$variance) # MAE res <- residuals(fit) mae <- mean(abs(res)) # MAPE pt <- (res)/y mape <- mean(abs(pt)) # MSE (Mean Squared Error) mse <- mean(res^2)
The performance was:
RMSE: 71.73491 MAE: 51.70627 MAPE: 0.02833748 MAE: 5145.897
Where the average travel time in London was around 1700 seconds.