4
$\begingroup$

Suppose I want to predict Amazon or Netflix demand, using demand data over the past year. For example, I might want to forecast the number of sales in the Electronics category on Amazon, or the number of times someone wants to rent Titanic on Netflix. My dataset consists of daily demand per item over the past couple of months, along with item metadata (tags and categories), split by things like customer demographics (age group, gender, location, browser, job -- some of these might be unknown).

To be concrete, let's suppose I want to forecast the number of times someone wants to rent a Comedy on Netflix, and I want to make this forecast at various levels (e.g., overall, by the state the customer lives in, by male/female, etc.). How would I go about this?

My naive first thought is to form a time series at each level I care about (e.g., form a time series of comedy demand by all the males living in Florida), and build some kind of time series model on top of this (I guess an ARIMA model...?). But this seems wrong for a bunch of reasons (not only would I be building a ton of different models for all the different possible levels, but each level would be ignoring a lot of data from closely related levels).

Any suggestions? Surprisingly, I couldn't find any papers related to this problem when Googling, but I might just be using the wrong search terms. (I learned a smidgen of time series analysis a couple years ago, but I was incredibly bad at it.) Also, I'm interested in both methods (what algorithms to use) and particular statistical libraries that might be useful (e.g., R packages or Python libraries).

$\endgroup$
4
$\begingroup$

If you do a good enough job modeling the important predictor variables, you probably will not need to worry as much about the time series aspects (You should probably still test for serial correlation and adjust for it if needed).

Most of the times series style association you will see can easily be modeled by things like day of the week, holiday/vacation indicators, and time since the dvd release or some form of advertising or event that spurs rentals of a particular movie.

$\endgroup$
  • $\begingroup$ this is a bit late!, but could you explain more how to model the predictor variables? For example, suppose I want to make forecasts at daily granularity of the # of times somebody in Florida rents a comedy, and my predictors are the ones you mentioned. Ignoring ts aspects like serial correlation, are you suggesting I form a dataset consisting only of the Florida-comedy rentals (i.e., ignore all other rentals in Florida like horror rentals, and ignore comedy rentals by other states), and build my model off this? So for each data slice, I build a new and independent model? $\endgroup$ – raegtin May 22 '11 at 20:34
  • $\begingroup$ In other words, I guess my question had two parts: 1) how to deal with the hierarchical nature of the problem (I want forecasts at different slice granularities, so when making forecasts for slice X, should I care that I have data outside of slice X that is nonetheless related to it?) and 2) how to reconcile the fact that each individual time series slice has both time series components and non-time series components. It sounds like you're suggesting that for 2), the time series parts are less important than the non-time series parts, and for 1), I ignore the hierarchical/interrelated nature? $\endgroup$ – raegtin May 22 '11 at 20:41
3
$\begingroup$

When you have a number of Endogenous series possibly cross-related and a number of Exogenous series, this is referred to as a VECTOR ARIMA MODEL , a super-set of a VAR model and an ARIMA MODEL. We have resolved Daily forecasts for a Family TYpe and Daily forecasts for "children" or "subset categories" by incorporating the Parent as a possible predictor variable to model each of the "children" speaking to your concern about "ignoring data". VARIMA models are not really practical because each series might have different dependencies on level shifts and local time trends. My suggestion is that you form "reasonable families" , model both the Parent and the Children using ARMAX models and then use a reconciliation strategy.

$\endgroup$
  • $\begingroup$ @Greg: I must seriously take issue with your dismissive remarks about the role of time series analysis. Unfortunately ignoring the "time series complication" has serious side effects that may either falsely reject or falsely accept hypothesis for models that are under-specified i.e. those that ignore the ARMA COMPONENT IN THE ARMAX MODEL. There is no such thing as a free lunch when it comes to modelling correctly. $\endgroup$ – IrishStat May 8 '11 at 22:27
  • $\begingroup$ Sorry, I did not mean to sound dismissive (and have edited to clarify). My thoughts were more to the point that if you model using only time series without covariates then I would expect certain lags to be very significant (7 days could be likely due to weekend effects), but if those covariates are included, then those lags may become completely unimportant (or not). I would focus on the covariates first. $\endgroup$ – Greg Snow May 9 '11 at 15:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.