I have had success with integrating hourly dummies into the model which effectively forecasts what is expected at hour j based upon the previous day's hour j value rather than the auto-regressive structure using the value at j-1 etc . This easily extends to incorporating day-of-the-week effects , week-of-the=year effects , monthly effects , pre and post holiday effects , long-weekend effects. This approach often is sufficient but sometimes needs/incorporates an ARMA modification
EDIT UPON RECEIPT OF 835 DAYS OF DATA FOR 24 HOURS:
A very workable solution for this multi-frequency data set (24 hours and 7 days) possibly holiday/event/promo driven is available in a piece of software that I have helped write (AUTOBOX). I will try to offer an honest description of how the solution unfolds. One can not simply apply ARIMA to the entire series (20,040 data points) as the first observation in each day is primarily driven/related to the first observation 1 day ago and the particular day-of-the-week , week-of-the-year, week-of-the-month, month-of-the-year etc while an ARIMA model would falsely use short-term prior hourly values and miss the big picture . Additionally the underlying model for this kind of daily data is primarily deterministic not simply auto-regressive.
The first step is to build a model for the total daily series (GROUPT)
shown here
which uses monthly indicators and day-of-the=week indicators and level shifts in conjunction with outlier detection. The second step is to do this for each of the 24 periods using the daily total as possible important predictor reflecting high-level trends. I show here the results for hour10 using monthly dummies , day-of-the-week dummies , step/level shifts AND the GROUPT reflecting macro trends/effects.
using
. We now have 24 individual forecast vectors and we then introduce an option to reconcile them with the forecast of GROUPT. I present here two possible reconciliations . Top-down
and bottom-up
for the future hourly forecasts.
Notice that there is no spurious auto-correlation induced by the consecutive zeroes in this approach.
MODIFIED TO INCLUDE MODEL RESULTS FOR HOUR 10:
