As Gardner notes "almost all point forecasts are wrong", so prediction intervals (PI) are necessary to quantify uncertainty and help us make informed decisions. There exists theoretical PI, and in some instances they work. In many instances there is no theoretical PI, for example if you are combining multiple models. A review of literature especially for time series forecasting, I found the following procedure from Taylor and Bunn that uses quantile regression on the error from the fitted value to generate empirical prediction interval.
A small toy example in
R programming language replicating the above method for 90% PI:
library("forecast") library("quantreg") ## Fit the forecast model and obtain error x <- AirPassengers arm.mod <- auto.arima(x) arm.fit <- fitted(arm.mod) error <- AirPassengers-arm.fit ## Quantile Regression to obtain 90% pred interv based on Taylor and Bunn (1999) t <- 1:length(x) tsq <- t^2 quant.05 <- rq(error~t+tsq,tau=0.05) quant.95 <- rq(error~t+tsq,tau=0.95) ## Make Predictions for quantile regressions t <- 1:12 tsq <- t^2 newdata <- as.data.frame(cbind(t,tsq)) pred.05 <- predict(quant.05,newdata) pred.95 <- predict(quant.95,newdata) ## Estimate Empirical Pred Interval forecast <- forecast(arm.mod,h=12) forecast.lpl <- forecast$mean+pred.05 forecast.upl <- forecast$mean+pred.95 forecast.limit <- cbind(forecast$mean,forecast.lpl,forecast.upl) ts.plot(forecast.limit)
Below are my questions:
- Should I use the error from the fitted model in Quantile Regression or should I use out of sample forecast errors ?
- Is my understanding (from the
Rprogram) of the method outlined in the article correct?
- When I plot the empirical prediction interval it seems very narrow ?
- Are there any other empirical PI methods other than the one specified above for time series forecasting ?
Very narrow PI: