4
$\begingroup$

I understand the the difference between these three types of intervals but I will summarize briefly:

  • Confidence Interval: an interval that will contain the true mean value say, 95%, of the time.

  • Prediction Interval: an interval that will contain 95% of all response values for a single new observation.

  • Tolerance Interval: an range that will contain a given proportion of the population 95% of the time.

My question is in regards as to which I would use when making a prediction. For the sake of a useful example, perhaps I am predicting the winner of an election. Which interval would be most statistically correct and beneficial to use?

$\endgroup$
2
$\begingroup$

This will depend on what you want to use the forecast and the interval for.

For instance, I do forecasting for retail, and our prediction intervals (more precisely: high quantile forecasts) are used for replenishment. In such a use case, the relevant thing is, say, a 95% quantile forecast, because this will translate more-or-less into a specific service level, which will hopefully balance the costs of understock and overstock.

As an example, let's look at the AirPassengers dataset. Maybe we want to provide enough capacity to achieve a service level of 95%. We can do this by fitting a model and looking at a 90% prediction interval. This is comprised of the 5% and the 95% quantile forecasts. Below, the "Lo 90%" is the number such that we expect a 5% chance of observing fewer passengers, while the "Hi 90%" is the number such that we expect a 95% chance of observing fewer passengers - together, the two numbers bracket off a prediction interval that we expect a 90% chance of covering the next observation.

> library(forecast)
> (foo <- forecast(auto.arima(AirPassengers),h=1,level=0.90))
         Point Forecast   Lo 90    Hi 90
Jan 1961       446.7582 427.413 466.1034
> plot(foo)

forecast with prediction interval

So, to achieve 95% service level, we would plan for providing capacity for 466.1 (thousand) passengers. (I'm glossing over different possible definitions of the service level, which here don't really make a difference. Plus, in planning for a longer lead time, we would need to take remaining autocorrelation into account, and so forth.)

Most forecasters I work with are only interested in prediction intervals, because you can verify them to a certain extent, simply by observing the next realization. Confidence intervals can never be verified, and you will have to trust that you have a correctly specified model for the mathematics to work. (And I have never seen the tolerance interval used in forecasting.)

When you are predicting the winner of an election, a prediction interval does not really make sense, because this random variable is not numeric. Unless you are not predicting the winner, but, say, a candidate's share of the vote.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you! Can you go into more detail explaining the quantile forecast? Maybe even using some R code as well (I saw you also use R). $\endgroup$ – a.powell Jul 24 '16 at 13:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.