Real future value out of 95% predict interval for ARIMA forecast

Question

When the predicted future value actually materializes, does it matter if it is outside the predict interval?

The value of prediction intervals is that they express the uncertainty in the forecasts. If we only produce point forecasts, there is no way of telling how accurate the forecasts are. However, if we also produce prediction intervals, then it is clear how much uncertainty is associated with each forecast. For this reason, point forecasts can be of almost no value without the accompanying prediction intervals.

-- from Hyndman & Athanasopoulos (2018) OTexts

I have read the definition of predict interval from the book, but I still not sure about the value outside the predict interval. Can I say it is a warning signal or structure change?

The example below

set.seed(123)
ts.sim <- arima.sim(list(order = c(1,0,0), ar = 0.8), n = 200)
ts.plot(ts.sim)

model <- Arima(ts.sim,c(1,0,0),include.mean = FALSE,include.drift = FALSE,method = "ML")
plot(forecast(model,h=20),ylim=c(-8,8))

lines(c(rep(NA,200),5+rnorm(20,0,1)),type="l",col="red")

The red line are out of the 95% predict interval.

Thanks for every help!

Answer 1

Your question is whether the points outside the prediction interval indicate a warning signal or structure change. While I am not sure about what you mean by "warning signal" and by "structural change", I figure that you want to know how to interpret data outside the prediction interval.

Well, let's say your prediction interval should contain the new value with a probability of 0.95. Then that means, that you should see about one out of 20 points to be outside the interval, and that would be nothing to worry about. But If there are, in some time interval, clearly more than 5% outside the prediction interval, then this is probably a "structure change". So the red line you plotted would definitely be something to worry about.