I am working on a problem where I want to forecast multiple time series. We have time series data in the following format:

ID Income Jan16 Feb16 Mar16 ... Dec18 1 2 200 300 250 600 2 1 450 450 300 1200 3 2 390 950 400 200 4 3 120 350 200 6100

In reality, our data spans millions of individual IDs and about 6 years of time.

As you can see, each row represents an individual customer ID and each column represents that customers value for a given month. Our goal is to use this data to create a forecast for each user ID that extends 6 months out.

The issue that I'm running into is that my friend is treating this as a regression problem, by making a feature set from Jan16-Nov16 (plus the income feature) and using those values to predict Dec18. He has created multiple Feed Forward Neural Net models, using a sliding window of time on the feature set (e.g., Feb16-Jan17 to predict Dec18, Mar16-Feb17 to predict Dec18). He is then averaging those predictions over about 30 models to create a forecast for Dec18. He is then training separate models to predict Nov18, Oct18 to create an overall extended forecast for each ID.

In my mind, this is the wrong way to approach the problem because of time effects, but I'm not well versed in how Feed Forward Neural Nets handle autocorrelation. In my mind, we should be fitting each univariate forecast separately, but that doesn't take into account the variability in the income feature. What can I tell my friend to help him understand that his approach isn't correct? What alternative can I suggest/is more proper?

  • 1
    $\begingroup$ Lots of things you could try. But first, your friend should assess forecast error, such that you can quantify how good or bad a given alternative approach is. $\endgroup$ – generic_user Mar 13 '19 at 18:15
  • $\begingroup$ stats.stackexchange.com/questions/380599/… is a good place to start. Note that if you had income over time it would be one of the X's in the ARMAX model. Also note that one of our large telecommunications clients are routinely forecasting millions of lines $\endgroup$ – IrishStat Mar 15 '19 at 13:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.