6
$\begingroup$

I'm trying to do some really simple Time Series work. I started off looking at monthly counts but the more I thought about it (and the more I read posts by IrishStat) the more I realised my view of this was fundamentally flawed. My main reason for thinking this is the plethora of timing issues there are with using monthly aggregate figures:

  • Varying month length
  • Varying numbers of weekdays
  • Weekends
  • Holidays

But what gets me is that this must be a really common issue, I mean practically every big company out there must have an executive report that includes monthly figures. How does one balance for the discrepancies between the months you are comparing?

I'm using R to slowly crawl through this stuff and currently I'm just looking at cross correlations with lag. I've looked at decomposition and how to adjust for seasonality but how does one adjust for dates that you know will have an effect on business activity? Is there a way to integrate a list of dates that should be treated differently or somehow compensate for the number of weekend/holiday days in a month (or in a week, or in a year)?

A general explanation of the concept would be much appreciated.

Edited to make the question more succinct:

I am looking at a time series of monthly counts. 2 examples from this time series would be:

  • February 2012 is 29 days long, has 8 weekend days and zero public holidays
  • May 2013 is 31 days long, also has 8 weekend days but also has 2 public holidays

How would I compare them without having my conclusions adversely affected by their differences?

  • I could divide by days in month but this doesn't take into account holidays
  • I could divide by working days in month, but this doesn't differentiate between a holiday and a weekend.
$\endgroup$
  • $\begingroup$ What exactly are you asking with "How does one balance for the discrepancies between the months you are comparing?"? Is this about how different months have a different number of days? If so, you could divide the monthly observations by the number of days and treat that as your data. Are you asking something different? $\endgroup$ – Macro Feb 19 '13 at 16:50
  • $\begingroup$ I am asking that but I think the issue also encompasses holidays, number of weekdays etc. For instance, activity may decrease on weekends (but not stop entirely) so how does one balance this on top of balancing for month length? It might well be that you divide by weekdays in the month or something like that but then on top of this there is also the issue of dealing with a varying number of public holidays. I'll edit and make it more clear... $\endgroup$ – Tumbledown Feb 19 '13 at 16:57
  • 2
    $\begingroup$ plus consider that some public holidays move - easter, chinese new year, Ramadan, school holidays, etc. Easter is particularly annoying in my own version of your problem because it isn't even in the same quarter each year. $\endgroup$ – Peter Ellis Feb 19 '13 at 18:28
  • $\begingroup$ Also, what about one off events? Ranging from additional holidays as mentioned by Peter (such as Jubilee celebrations in the UK) to events such as warehouse fires, workforce strikes etc. $\endgroup$ – Tumbledown Feb 20 '13 at 8:37
  • $\begingroup$ @Tumbledown Intervention Detection would be used to identify the existence of events that had NOT been suggested by the user. These would then join with events that had been suggested by the user as possible predictive series. $\endgroup$ – IrishStat Feb 20 '13 at 13:16
2
$\begingroup$

You asked for a general explanation of the concept. Your comments about the current status of forecasting at three levels of aggregation is dead on ! My answer may not precisely deal with some of your specific interests as you have focused on some distractions but I thought that I would share the follwing with you. I was asked to discuss how software I had helped write could deal with and accomodate monthly vs weekly vs daily forecasts.

My response was in three parts : A. Overall comments on weekly versus monthly B. The argument for parsing the momrhly forecast to dai;ly using simple ratios C. The argument against #2 and FOR daily forecasts to be DIRECTLY developed and then used to make weekly and/or monthly forecasts.

Response A)

Monthly:

Advantages – Fast to compute, easier to model, easier to identify changes in trends, better for strategic long term forecasting

Disadvantages – If you need to plan as the daily level for capacity, people and spoilage of product then higher levels of forecasting won’t help understand the demand on a daily basis as a 1/30th ration estimate is clearly insufficient.

Causal variables that change on a frequent basis (ie daily/weekly – price, promotion) are not easily integrated into monthly analysis

Integrating Macroeconomic variables like Quarterly Unemployment requires an additional step of creating splines.

Weekly:

Advantages – When you can’t handle the modeling process at a daily level you “settle” for this. When you have very systematic cyclical cycles like “artic ice extents” that follow a rigid curve and not need for day of the week variations.

Disadvantages – Floating Holidays like Thanksgiving, Easter, Ramadan, Chinese New Year change every year and disrupt the estimate for the coefficients for the week of the year impact which CAN be handled by creating a variable for each.

The number of weeks in a year is subject to change and creates a statistical issue due to the fact that every year doesn’t have 52 weeks. We have seen the need to allocate the 53rd week to a “non-player” week to make the data a standard 52 week period which is workable, but disruptive compared to daily data.

Causal variables that change on a frequent basis (ie daily/weekly – price, promotion) are not easily integrated into monthly analysis

Integrating Macroeconomic variables like Quarterly Unemployment requires an additional step of creating splines.

Response B) ( tongue-in-cheek answer )

Assuming you had the daily data in a data warehouse and you wanted to develop daily from the monthly forecasts.

I would take monthly forecasts and partition it to daily in the following manner.

  1. Compute daily averages from the history database thus D1,D2,….D7 averages are known and will be used I would compute the overall average (XBAR) and compute 7 indices I1=D1/XBAR ; I2=D2/XBAR …. I7=D7/XBAR thus the 7 I’s represent percentages i.e.

.9,1,2,…..8 for example.

  1. I would then compute a forecast for DAY1 in the month by using the appropriate I value and get [1/30]*Monthly forecast*I , essentially adjusting the baseline daily forecast of 1/30 th of the monthly expectation.

  2. Finally I would then normalize these DAILY forecasts so that they add to the monthly forecast.

Response C)

I should also add that the procedure I laid out in (B) is subject to a number of assumptions regarding the historical data , most of which are unrealistic in my opinion:

1) That there are no trends and no level shifts . 2) That there are no PULSES ( one time unusual values ) 3) That there are no Holiday effects OR special days in the month effects OR special weeks in the month effects or beginning/end-of the month effects 4) There are no seasonality effects (monthly or weekly ) 5) There have been no changes in the day-of-the-week averages over time 6) There is no autoregressive structure
7) There have been no chnages in model paramters or the error variance over time.

All of these considerations suggest that models should be developed at the daily level in order to provide information as quickly as possible.

Hope this helps !

$\endgroup$
  • $\begingroup$ Very interesting, thanks! As I say, this level of analysis is quite new to me but all the information in your answer is very useful. We're going to give a daily level analysis a go and see how we get on. $\endgroup$ – Tumbledown Feb 27 '13 at 9:08
  • $\begingroup$ @Tumbledown Thanks for your nice words. It is nice to know that you are appreciated.If you need help with daily modelling you can contact me and I will try and pass on pointers to you as to how to proceed and some of the pitfalls you might encounter on the way. $\endgroup$ – IrishStat Feb 27 '13 at 14:35
0
$\begingroup$

It will depend on your research question.

I'm interested that there is no answer to this question so far; does it mean there is no standard solution? Seasonal adjustment seems to be extremely ad hoc...

If the company you are looking at is focused on "working days" (eg they are a firm that only produces on those days) then counts per working day would make sense. However, if business takes place even on holidays and weekends you would probably be better off with just counts per day. In either case you might find that in any modelling you do you want to include the number of weekends and/or non-working days as a variable to see if it helps explanation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.