3
$\begingroup$

EDIT (Tue12Mar): The bug fixes are on their way. See https://github.com/steve-the-bayesian/BOOM/issues/25

I've been working with bsts to model time-series and make forecasts, but have run into some problems with the predict function and holidays.

When I train the model with regressor holidays these regressors do not appear to be used in the predict function. For instance in a simple local level model the holiday regressor causes the response variable to go down, but the predictor of the response over the same period does not capture this behaviour.

I attach below a close to minimum working example based on the regression.holiday code from the documentation. You will see the green line representing the mean/95%CI failing to capture the Memorial Day holiday Plot of predict failing to capture holiday

EDIT (Mon4Mar18): Here's a .csv with the simulated data. Consists of 3 columns: Date, Value and Train (TRUE/FALSE depending whether in training set)

library(bsts)
library(tidyverse)
set.seed(12345)

trend <- cumsum(rnorm(1095, 0, .1))
  dates <- seq.Date(from = as.Date("2014-01-01"), length = length(trend), by = "day")
  y <- zoo(trend + rnorm(length(trend), 0, .2), dates)

AddHolidayEffect <- function(y, dates, effect) {
  ## Adds a holiday effect to simulated data.
  ## Args:
  ##   y: A zoo time series, with Dates for indices.
  ##   dates: The dates of the holidays.
  ##   effect: A vector of holiday effects of odd length.  The central effect is
  ##     the main holiday, with a symmetric influence window on either side.
  ## Returns:
  ##   y, with the holiday effects added.
  time <- dates - (length(effect) - 1) / 2
  for (i in 1:length(effect)) {
    y[time] <- y[time] + effect[i]
    time <- time + 1
  }
  return(y)
}

## Define some holidays.
memorial.day <- NamedHoliday("MemorialDay")
memorial.day.effect <- c(-.75, -2, -2)
memorial.day.dates <- as.Date(c("2014-05-26", "2015-05-25", "2016-05-25"))
y <- AddHolidayEffect(y, memorial.day.dates, memorial.day.effect)

## The holidays can be in any order.
holiday.list <- list(memorial.day)

## Let's train the model to just before MemorialDay
cut_date = as.Date("2016-05-20")
train_data <- y[time(y) < cut_date]
test_data <- y[time(y) >= cut_date]
ss <- AddLocalLevel(list(), train_data)
ss <- AddRegressionHoliday(ss, train_data, holiday.list = holiday.list)
model <- bsts(train_data, state.specification = ss, niter = 500, ping = 0)
## Now let's make a prediction covering MemorialDay
my_horizon = 15
## Note adding the time stamps here doesn't help either
pred <- predict(object = model, horizon = my_horizon)
## Make a data frame for plotting
plot_info <- data.frame(Date = time(y), 
                        value = y, 
                        predict_mean = NA,
                        predict_upper = NA,
                        predict_lower = NA
                       )
plot_info[plot_info$Date %in% time(test_data)[1:my_horizon],]$predict_mean = pred$mean
plot_info[plot_info$Date %in% time(test_data)[1:my_horizon],]$predict_lower = pred$interval[1,]
plot_info[plot_info$Date %in% time(test_data)[1:my_horizon],]$predict_upper = pred$interval[2,]
## Let's make a pretty plot to demonstrate the problem
filter(plot_info, Date > time(test_data)[1] - 25 & Date < time(test_data)[my_horizon] + 10)  %>% 
    ggplot(aes(x = Date, y = value)) +
    geom_line() +
    geom_line(aes(y = predict_mean), col = "Forest Green") + # The prediction
    geom_line(aes(y = predict_lower), col = "Forest Green", lty = 2) + # lower bound
    geom_line(aes(y = predict_upper), col = "Forest Green", lty = 2)  # upper bound
$\endgroup$

1 Answer 1

0
$\begingroup$

I don't use bsts so i can't precisely tell what is the flaw be it simulation or analysis. Post your data in a csv file and I will try to help you.

Take a look at https://autobox.com/capable.pdf (slide 49) for some pointers on daily data and in general https://stats.stackexchange.com/search?q=user%3A3382+daily+data

edited after receipt of data (1095 observations)

The plot of the data enter image description here suggests that the simulation did not totally work as a number of interventions around memorial day are visually obvious. This is perhaps due to the effect of the ARIMA structure not being properly introduced in the simulation

AUTOBOX ( which I have helped to develop) automatically identified 1) the underlying arima model (1,1,0)(0,0,0)7 with coefficient = .413 AND 2) the following list of interventionsenter image description here with a curious suggestion of anomalies primarily around Memorial Day AND 3)A suggested model model of enter image description here and here enter image description here & enter image description here and 4) an estimated effect of -1.99 for Memorial Day. Note well that Holiday Effects are often over a period of days NOT JUST on the actual holiday. AUTOBOX identifies lead and lag effects if they exist ... not so in this case as only the actual holiday was effected as a result of the simulation.

Hope this helps you and the author of the software you are using. Your local level model is clearly inadequate . I didn;t see where you actually suggested the ARIMA model form when you performed the simulation but again I don't know bsts .

$\endgroup$
3
  • $\begingroup$ Thanks for your help! I've added a .csv link to a google drive file. Let me know if there are any problems downloading. $\endgroup$ Mar 4, 2019 at 13:57
  • $\begingroup$ Thanks again for the detailed reply. I will have a look at the software. The simulated data are intended to capture the general problem I'm facing in real data in a minimal way, with a single holiday effect that seems correctly caught for in-sample predictions, but not for out-sample predictions. I've tried a variety of approaches, including ARIMA, with the real data I'm working on, but they haven't performed as well as the structural time series models I've tried. The pseudo-model selection approach with slab and spike priors helps our problem a lot since we have a lot of potential regressors $\endgroup$ Mar 5, 2019 at 7:33
  • $\begingroup$ if you need any special help offline feel free to contact me as I am very familiar with daily data. $\endgroup$
    – IrishStat
    Mar 5, 2019 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.