The Problem

I have been tasked with a similar problem to that described in Forecasting hourly time series with daily, weekly & annual periodicity. My data shows the number of times that one of 384 different events has occurred per hour between 01/01/2005 and 31/12/2013. I have been asked to provide a 28 and 84 day forecast showing the number of each type of event expected per hour.

My Experience

Before I go on I will clarify that I am a bit of a newbie in R (and statistics) and while I realise that this is not at all a simple task (nor possibly one that R is best suited to) I don't have access to any other tools and there is no one else in my organisation that is prepared to tackle this problem.

What I have tried so far

So I have restricted myself to just one event type and have tried using auto.arima and HoltWinters on the entire 75,000 hourly datapoints using timeseries ts(data,frequency=8760), ts(data,frequency=8760/12), and ts(data,frequency=8760/52.1xxxx). I excluded ets because it doesn't cope with frequencies above 24. The results from these tests were pretty much flat line which isn't suitable for what I want to use the forecast for.

Because each week is closely related to the previous week/s I tried to simplify things by only forecasting based on the previous 4 and 8 weeks using ts(data, frequency=168) (treating it as weekly data).

The results from the HoltWinters experiment look promising and I have discarded ARIMA because it is still producing a straight line. Over a 28 day period in 2013 the total of the HoltWinters "forecasted" values based on the previous 4 weeks is within 10% of the actual values which is close but I think I could get closer. (This is probably not the best way to test the values but it seems like a good indicator.)

My Planned next steps

I am now wanting to try and factor in the same period for the year/s previous. That is, if I am forecasting for December 2013 I will use November 2013 and December 2012, 2011, etc.

The Questions:

  1. How do I forecast using the previous weeks and the same period for the previous years?

  2. Is there a better way of doing this? (I am open to suggestions here, I am really only getting by, by testing and refining my method and am conscious of the fact that there is probably much better ways of doing this.)

  3. It is quite likely that occurrences of different event types are linked (i.e. if event a is happening a lot event b will likely be happening a lot too, and event c will probably not happen). As a next step is it possible be find and then factor in these correlations?

Sorry that I have not been able to attach my data but the computer system at work does not allow us to access places like dropbox and I cant see any other way of attaching a csv to this question. If it is required to answer my questions then I will take a copy home and upload it from there.

  • 1
    $\begingroup$ Use tbats(). It is ideally suited to this problem. $\endgroup$ May 2, 2014 at 3:20
  • $\begingroup$ @Tim Can you attach your data to the question? The link in the linked question no longer works. $\endgroup$ May 4, 2014 at 7:06
  • $\begingroup$ @RobHyndman Thanks for the reply. I have had a look at 'tbats()' but I am basically getting the same or similar results to what I was getting with 'auto.arima'. I am not convinced that I am using it properly. Are there any tutorials around that explain how to use it? A quick Google search didn't yield anything that I could follow. $\endgroup$
    – Tim
    May 5, 2014 at 2:10
  • $\begingroup$ @fgnu I will try and upload some of the data soon. I will need to get it cleared and then take it home to upload it. $\endgroup$
    – Tim
    May 5, 2014 at 2:11
  • $\begingroup$ There is an example with daily data at robjhyndman.com/hyndsight/dailydata which might help. Just add one more level of seasonality for hourly data. $\endgroup$ May 5, 2014 at 2:19


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.