I have seen many kernels that are using machine learning algorithms (Xgb, LSTM, others) on time series forecasting.

A time series data typically has trend and seasonal components. In general my question is

  1. Is it necessary to remove trend and seasonality (i.e make it stationary ) before applying machine learning/supervised learning (Xgb, LSTM, others) algorithms for time series data ?

  2. when will machine learning/supervised learning (Xgb, LSTM, others) algorithms for time series data gives good result? When will they not give good result?

  3. Any guidelines for using machine learning/supervised learning algorithms for time series data ?

  4. If there is seasonality and trend how to tackle the problem? One way is to detrend and remove seasonality, and then use ML algorithm for forecasting.
    Are there any other approaches especially if time series has trend?

  5. Finally, How will you verify the forecast results? I mean, can we look at the residuals/something to infer forecast results makes sense.

  • $\begingroup$ Your should definitly check freerangestats.info/blog/2016/11/06/forecastxgb $\endgroup$ Commented Jan 7, 2019 at 14:48
  • $\begingroup$ As some more information in machine learning approaches vs. traditional statistical ones, the very recent M4 competition just closed this year and I highly encourage you to take a read on the results: researchgate.net/publication/… In summary, all "pure" ML approaches did not perform well at all. Out of the top five models, I believe four were ensembles of classical methods, and first place was a hybrid neural network + Holt-Winters multiplicative $\endgroup$
    – aranglol
    Commented Jan 7, 2019 at 20:30

1 Answer 1

  1. "ML" approaches will typically do better in high signal to noise situations, and with enough data. With less data, the traditional forecasting approaches are often superior.

  2. This is a very broad question. One recommendation: familiarize yourself with the state of the art in "classical" forecasting. I recommend the excellent free online book Forecasting: Principles and Practice (2nd ed.) by Athanasopoulos & Hyndman. Test your ML forecasts against simple benchmarks, which surprisingly often outperform more complex algorithms.

  3. One way to deal with trend is to include a linearly increasing predictor and extrapolate that out. Forecasters often find that dampening the trend improves forecasts. Similarly, you can include periodic functions like harmonics or bump functions as predictors to model seasonality.

  4. There are many accepted measures of forecast accuracy. You may also want to look through the tag wikis on the MAPE, the MASE, the MAE or the MSE, or through threads tagged both "forecasting" and "accuracy".

  • $\begingroup$ Can you explain 3? $\endgroup$
    – v09
    Commented Jan 7, 2019 at 16:50
  • 1
    $\begingroup$ For trend, include a predictor that counts the seconds, days or whatever time granularity you have, starting at an arbitrary beginning of time. Both historically and for the future. For seasonality, transform this into multiple periodic functions whose periodicity is a multiple of your seasonality base periodicity. E.g., for intra-yearly seasonality on daily data, include one sine wave with period 365.25, one with 182.6, one with 121.8 etc. $\endgroup$ Commented Jan 7, 2019 at 16:58
  • $\begingroup$ Thanks for reply. According to the the above comment, it is not always mandatory to make your time series stationary before applying ml algorithm to time series forecasting. Am I right? $\endgroup$
    – v09
    Commented Jan 7, 2019 at 17:43
  • 2
    $\begingroup$ Stationarity is a tricky thing. "Plain vanilla" ML implementations, e.g., LSTMs, that assume stationarity will of course require stationarity. But it's straightforward to model certain kinds of nonstationarity without transforming the input. For instance, you can model trends and seasonalities - two kinds of nonstationarity - in the way I described. So the question is really what assumptions your algorithm of choice makes. $\endgroup$ Commented Jan 7, 2019 at 19:32
  • 1
    $\begingroup$ @tjt: sorry, I don't know enough about LSTMs to answer that with any confidence... $\endgroup$ Commented Oct 29, 2020 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.