2
$\begingroup$

In order do make a regression for daily sales data i need to set up different dummies (e.g. day of the week, monthly, yearly, week of the year, moving holidays...)

dummies <- cbind(model.matrix(~template$Weekday)[,2:7],
                 model.matrix(~as.factor(template$Month))[,2:12],
                 model.matrix(~as.factor(template$Year))[,2:5],
                 model.matrix(~as.factor(template$CalendarWeek))[,2:53])
colnames(dummies) <- c('Tue','Wed','Thu','Fri','Sat','Sun',
                   'Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec',
                   paste0('y',rep(2:5)),   #5 years of data in total
                   paste0('w',rep(2:53)))   #calendar weeks

"template" is a data.table which includes all the information (e.g. Sales, Date, Weekday...)

Right now i am missing holiday dummies (including lead & lag dummies). In order to catch the moving holidays i have to create one dummy per holiday.

How do i figure out how many lead and lag dummies i need for different holidays? Is there a way to create them "on the fly" meaning i start adding lead/lag variables step by step (e.g. first: Easter-1, second: Easter-2...) and check each time if my regression model improves (e.g. AIC goes down)?

How do i deal with not-moving holidays? Simply creating dummy variables for "day of the year"? E.g. 366 dummy variables

Are there any kind of dummy variables missing so far? E.g. day of the month?

Thanks for your support!

Update:

Sample data

Improve this question
$\endgroup$
3
  • $\begingroup$ Post your data to dropbox.com and specify the beginning date and what country the data is from. $\endgroup$
    – Tom Reilly
    Commented Sep 30, 2015 at 14:36
  • $\begingroup$ Hi @TomReilly, i added the data under Update. Main country is certainly the US but it is possible that there are influences of other countries (talking about specific holidays) in the data - it is coming from an international supply chain... $\endgroup$
    – RandomDude
    Commented Sep 30, 2015 at 18:22
  • $\begingroup$ Hi...the link to the data doesn't work anymore...can you fix and I will post the whole model and results to dropbox. $\endgroup$
    – Tom Reilly
    Commented Oct 16, 2015 at 12:30

1 Answer 1

Reset to default
1
$\begingroup$

This is not the full model. There are a total of 74 variables. It looks like you have already have been drinking at the trough of daily data. What is there to be learned differently from your question here that wasn't answered here? Decomposition of daily time series (several years) with multiple seasonal patterns

enter image description here

Improve this answer
$\endgroup$
10
  • $\begingroup$ Thanks for your answer. Well i already accepted that there is no "simple" way to forecast daily data + there is a reason why software solutions like autobox exist. But for the sake of my thesis i have to come up with some model to forecast daily data. So i am comparing different approaches available in R (e.g. tbats, regression,...) and compare the results of different datasets that i have to forecast in order to suggest the "least worst" model. $\endgroup$
    – RandomDude
    Commented Oct 1, 2015 at 12:05
  • $\begingroup$ So i am trying to improve the regression approach as much as i can. I set up dummy variables and afterwards use variable selection to find a final regression model. Would you suggest any other dummy variables that i can/should implement: Day of the week, Month, Calendar Week, Day of the year, Moving holidays (including lead and lag)? Don't think i can implement Level Shift or Pulse, at least i am not aware of any method. $\endgroup$
    – RandomDude
    Commented Oct 1, 2015 at 12:11
  • 1
    $\begingroup$ As you can see above, day of the week, month of the year were important, but not all of them. Yes, certain holidays and their lead and lags are important. Why don't you download a 30 day trial from autobox.com and use that as well? Day of the year is not something I have ever seen before. What could that be? Day of the month is something we have seen. $\endgroup$
    – Tom Reilly
    Commented Oct 1, 2015 at 12:16
  • 1
    $\begingroup$ No, there is no sense to think that a given year would be different unless there was some marketing or policy change. "Dummies for different years" is like a level shift variable or multiple ones. $\endgroup$
    – Tom Reilly
    Commented Oct 1, 2015 at 12:18
  • 1
    $\begingroup$ Already thought about using autobox, as soon as my model works i will download it so i can compare performance and also show that it makes sense to invest in such a software solution. $\endgroup$
    – RandomDude
    Commented Oct 1, 2015 at 13:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.