The Problem
I have been tasked with a similar problem to that described in Forecasting hourly time series with daily, weekly & annual periodicity. My data shows the number of times that one of 384 different events has occurred per hour between 01/01/2005 and 31/12/2013. I have been asked to provide a 28 and 84 day forecast showing the number of each type of event expected per hour.
My Experience
Before I go on I will clarify that I am a bit of a newbie in R (and statistics) and while I realise that this is not at all a simple task (nor possibly one that R is best suited to) I don't have access to any other tools and there is no one else in my organisation that is prepared to tackle this problem.
What I have tried so far
So I have restricted myself to just one event type and have tried using auto.arima
and HoltWinters
on the entire 75,000 hourly datapoints using timeseries ts(data,frequency=8760)
, ts(data,frequency=8760/12)
, and ts(data,frequency=8760/52.1xxxx)
. I excluded ets
because it doesn't cope with frequencies above 24. The results from these tests were pretty much flat line which isn't suitable for what I want to use the forecast for.
Because each week is closely related to the previous week/s I tried to simplify things by only forecasting based on the previous 4 and 8 weeks using ts(data, frequency=168)
(treating it as weekly data).
The results from the HoltWinters
experiment look promising and I have discarded ARIMA because it is still producing a straight line. Over a 28 day period in 2013 the total of the HoltWinters
"forecasted" values based on the previous 4 weeks is within 10% of the actual values which is close but I think I could get closer. (This is probably not the best way to test the values but it seems like a good indicator.)
My Planned next steps
I am now wanting to try and factor in the same period for the year/s previous. That is, if I am forecasting for December 2013 I will use November 2013 and December 2012, 2011, etc.
The Questions:
How do I forecast using the previous weeks and the same period for the previous years?
Is there a better way of doing this? (I am open to suggestions here, I am really only getting by, by testing and refining my method and am conscious of the fact that there is probably much better ways of doing this.)
It is quite likely that occurrences of different event types are linked (i.e. if event a is happening a lot event b will likely be happening a lot too, and event c will probably not happen). As a next step is it possible be find and then factor in these correlations?
Sorry that I have not been able to attach my data but the computer system at work does not allow us to access places like dropbox and I cant see any other way of attaching a csv to this question. If it is required to answer my questions then I will take a copy home and upload it from there.
tbats()
. It is ideally suited to this problem. $\endgroup$