This question is related to a previous post I've looked at (Calculation of seasonality indexes for complex seasonality), but deals with more granular data (daily instead of weekly), and transforming holiday seasons (instead of just holiday) creating a larger question about the reasoning behind dummy variables.

As an introduction, I'm looking to forecast daily retail sales for a highly cyclical e-commerce company. Each year, buying patterns shift for several reasons:

  1. Calendar shift (where, for example, 5/17/13 is a Friday, while 5/17/12 is a Thursday);

  2. Shifting of holidays set by the day of the week (i.e. Labor Day, Thanksgiving, Mother's Day);

  3. Shifts in the time and length of shopping seasons (i.e. in 2012, there were 32 calendar days, 21 workdays, and the season started on Nov. 28 in the holiday shopping season, while this year, there are 26 calendar days, 17 workdays, and the season starts on Nov. 29). These changes affect the overall shape and size of the seasonal curve, and not just the behavior at or near the holiday.

Among the retail forecasters I've encountered, there's a few options for dealing with forecasting shifting seasonality:

  • One is to only use exact calendar equivalent years to parcel out seasonality -- for example, 2013 exactly matched 2002 in terms of the placing of holidays. Then, you apply a 365-day ARIMA on the data. But, in e-commerce (and overall) a lot has happened since 2002 that affects how seasonality looks.

  • The second is exponential smoothing with dummy variables. But, as consumer behavior doesn't just change on the holiday (or the days approaching it), but relates to how far the holiday is in relation to other shifting dates (and the day of the week), you hit the problems of either creating a boatload of dummy variables for the distance from and to each holiday that can over-specify your model (and not make it easily generalizable to the year you want to forecast), or get a "general" seasonality that usually under-preforms. You could also do a fractional polynomial to approximate the curve by days until (or between) one holiday or another, but this creates the usual problems with fracpolys -- notably over-fitting and wildly strange results at certain points in and out of the sample.

  • The third option I've seen is to adjust the data from previous years to match the size of each holiday season by, in the case when the holiday shopping season is a week shorter than previous years, taking out a week and then adding back that revenue across the remaining season, or doing the reverse when the season is a week longer. After that's done, you can do a seasonal decomposition to parcel out the seasonality. The issue with this is that it involves a ton of subjectivity, and assumes that shoppers will evenly spread the lost week, rather than "back load" their behavior as the holiday more quickly approaches. Fixing the "back load" would then create even more subjectivity of when shoppers will start responding to the proximity to the holiday (such as Christmas).

So, I'm wondering the experience of other forecasters (especially those with retail experience) with this issue of shifting seasonality by day.

Looking forward to your thoughts. I can also add some dummy data if you want to see exactly what I'm talking about if you'd like.

  • $\begingroup$ You mention "daily" several times, but I still have to ask: do you really need daily forecasts? I'm not the expert on this, but it seems to me that you're creating a lot of over-fitting and noise issues by requiring daily forecasts. $\endgroup$ – Wayne May 29 '13 at 15:06
  • $\begingroup$ unfortunately, yes -- the model is used in part for staffing and purchasing decisions, which are done with daily point accuracy. $\endgroup$ – Bryan May 29 '13 at 18:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.