21
$\begingroup$

By default Prophet will return uncertainty intervals for the forecast yhat.

Unfortunately, the documentation about those "uncertainty intervals" is extremely vague, and it doesn't help that statisticians have precise definitions of both confidence intervals and prediction intervals (there is a big difference!), but not of "uncertainty intervals". In fact, the documentation could be taken to refer to either one, although a "prediction interval interpretation" sounds a little more likely to me.

Can anyone tell us (e.g., by digging through the code) whether these are confidence or prediction intervals?

$\endgroup$
3
  • 7
    $\begingroup$ +1 I wouldn’t mind hearing why the developers went with their unusual terminology, too. $\endgroup$
    – Dave
    Jun 27, 2023 at 10:42
  • 2
    $\begingroup$ @Dave: to be quite honest, I don't think I really want to know this... $\endgroup$ Jun 27, 2023 at 11:19
  • $\begingroup$ (if you haven't already) you can open a github issue and see what the developers say. $\endgroup$
    – qwr
    Jun 28, 2023 at 17:47

1 Answer 1

21
$\begingroup$

From digging through the code as you suggest it seems that they are prediction intervals. Specifically the model is fit by sampling a posterior with Stan (line 1249-1266 in prophet.R) and the interval is based on draws from the posterior predictive distribution. See line 1542 of the file prophet.R, which I reproduce below.

#' Prophet uncertainty intervals for yhat and trend
#'
#' @param m Prophet object.
#' @param df Prediction dataframe.
#'
#' @return Dataframe with uncertainty intervals.
#'
#' @keywords internal
predict_uncertainty <- function(m, df) {
  sim.values <- sample_posterior_predictive(m, df)
  # Add uncertainty estimates
  lower.p <- (1 - m$interval.width)/2
  upper.p <- (1 + m$interval.width)/2

  intervals <- cbind(
    t(apply(t(sim.values$yhat), 2, stats::quantile, c(lower.p, upper.p),
        na.rm = TRUE)),
t(apply(t(sim.values$trend), 2, stats::quantile, c(lower.p, upper.p),
            na.rm = TRUE))
  )

  colnames(intervals) <- paste(rep(c('yhat', 'trend'), each=2),
                               c('lower', 'upper'), sep = "_")

  return(dplyr::as_tibble(intervals))
}

See also the sample_model function that is called from sample_posterior_predictive, which shows exactly how the yhats are calculated on line 1573:

sample_model <- function(m, df, seasonal.features, iteration, s_a, s_m) {
  trend <- sample_predictive_trend(m, df, iteration)

  beta <- m$params$beta[iteration,]
  Xb_a = as.matrix(seasonal.features) %*% (beta * s_a) * m$y.scale
  Xb_m = as.matrix(seasonal.features) %*% (beta * s_m)

  sigma <- m$params$sigma_obs[iteration]
  noise <- stats::rnorm(nrow(df), mean = 0, sd = sigma) * m$y.scale

  return(list("yhat" = trend * (1 + Xb_m) + Xb_a + noise,
              "trend" = trend))
}
$\endgroup$
2
  • 1
    $\begingroup$ Perhaps worth pointing out that posterior predictive distribution - > prediction intervals relies on the assumption that the model is the true model. Put differently, if you create a model with Prophet that is missing key elements of the DGP, don't be surprised when the "prediction intervals" are poorly calibrated. $\endgroup$
    – Dan
    Feb 5 at 19:26
  • $\begingroup$ Yes I agree. Of course something similar will be true for any type of model; no kind of interval will save you from a bad model $\endgroup$
    – einar
    Feb 6 at 13:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.