How to treat holidays when working with time series data?

Question

I am analysing hourly demand data for electricity. To make my forecasts more accurate, how should I treat the national holidays in the data?

In particular, how should I treat them in R?

Answer 1

There is a little detail here, so a generic answer.

First of all, check if this problem exists; look at the residuals during holidays and test whether there is any significant problem with accuracy there. Your model might have already take holidays into account (for instance through some other predictors), or they are just irrelevant to what you trying to predict on your current accuracy level.

If the difference exists, try adding information about the holidays to the variables in the model; you can start with binary isHoliday, then think how to extend this to something more complex (i.e. add some adjacent days where it is handy to get a leave to enlarge the break, think of some continuous measure of "holidayness").

If your model is too dumb to use such variables, consider making two -- for normal days and for holidays.

Finally, if it happens you won't have to deal with predictions during holidays or it is a lesser problem to generate junk then, you may just throw this part of data.

Answer 2

I've always found it tough to handle the multiple (annual, weekly, daily) seasonality of electricity load/price data using time series methods. I use an approach (very) similar to IrishStat's except that I forecast the daily peak (MW) and total energy (MW/h) using machine learning methods (as opposed to time series), and then construct a linear regression model for each hour (1-24) of the day. The forecasted peak/total energy are features in each of the hourly models. The rest of the features are basically the same for all 26 models, with day of week, holiday and season represented as dummy variables. Obviously weather and lagged dependent variable values are also important features.

As an aside, six months of data is really not ideal regardless of your approach because this is clearly a process with an element of annual seasonality. Normally I'd say you need three years to properly train and test your model. With less than a year, you can't even fully assess those annual seasonality effects.

If you do go the time series route, just dumping the holidays will be a problem if you have a weekly seasonality term, e.g. if you try to explain the following Thursday's load as a function of the load on Thanksgiving.

Answer 3

When dealing with electricity data, I think the simplest option is to treat holidays as weekends (e.g. you have a dummy variable where 1 is a normal weekday, and 0 is a weekend or holiday). A more complicated option would be to have separate dummy variables (0/1) for weekday vs. weekend and normal day vs. holiday.

The holidayNERC function in the timeDate package is extremely useful in this situation. holidayNYSE is useful too.

Answer 4

I have done a lot of work with hourly data and have concluded that a two prong approach seems to deliver useful models. First of all we model the daily totals taking into account any day-of-the-week effects , any fixed-day of the month effects that can be identified along with any holiday effects. Each holiday can have it's own lead, contemporaneous and lag structure and there may be accompanying Friday before a Monday Holiday or Monday after a Friday Holiday effect. This model may also include local time trends , level shifts and of course one-time only events (pulses) and a possible need for either or both variance changes or parameter changes over time. Now with this model in place we can make daily forecasts for the future periods. The second step is to to construct 24 individual hourly models reflecting the incorporation of the daily total series. The reason for the 24 separate hourly models is the fact that consumption patterns during the day (intra) are often quite different across(inter) days. Each of the hourly models could have ARIMA structure reflecting historical usage for that hour and of course Level Shifts , Local Trends and Pulse effects. Individual hourly demand may or may not reflect daily total demand thus one needs to pay attention to that possibility. Since forecasts exist for the daily totals this can then be used to predict the hourly values.