13
$\begingroup$

Just a general question. If you have time series data, when is it better to use time series techniques (aka, ARCH, GARCH, etc) over machine/statistical learning techniques (KNN, regression)? If there is a similar question that exists on crossvalidated, please point me towards it--looked and couldn't find one.

$\endgroup$
1

4 Answers 4

11
$\begingroup$

Typical machine learning methods assume that your data is independent and identically distributed, which isn't true for time series data. Therefore they are at a disadvantage compared to time series techniques, in terms of accuracy. For examples of this, see the previous questions Ordering of time series for machine learning and Random forest is overfitting.

$\endgroup$
2
  • $\begingroup$ thanks for your response. To further your point, it seems that machine learning is more concerned on finding relationships in the data, whereas time series analysis is more concerned with correctly identifying the causes of the data--i.e. how stochastic factors are affecting it. Do you agree with this? $\endgroup$
    – Nagy
    Commented Dec 12, 2014 at 21:53
  • 5
    $\begingroup$ No, I would not agree with that summary. $\endgroup$
    – Tom Minka
    Commented Dec 13, 2014 at 22:28
5
$\begingroup$

As @Tom Minka pointed out most ML techniques assume iid inputs. There are a few solutions though:

  1. One can use all the past time series samples within the system 'Memory' as one feature vector, ie: x=[x(t-1),x(t-2),...x(t-M)]. However, this has 2 issues: 1) depending on your binning, you may have a huge feature vector 2- some methods require features within the feature vector to be independent, which isn't the case here.

  2. There exist many ML techniques which are specifically designed for such time-series data, for example Hidden Markov Models, which have been used very successfully for seizure detection, speech processing, etc...

  3. Finally, an approach I have taken is to use 'feature extraction' techniques to convert a dynamic regression problem (which has the element of time) into a static one. For example, the Principal Dynamics Mode (PDM) approach maps the input past feature vector ([x(t-1),x(t-2),...x(t-M)]) onto a static one ([v(1),v(2),..v(L)]) by convolving the past with a system-specific linear filterbank (the PDMs), see Marmarelis, 2004 book or Marmarelis, Vasilis Z. "Modeling methodology for nonlinear physiological systems." Annals of biomedical engineering 25.2 (1997): 239-251...

$\endgroup$
4
$\begingroup$

Francis Diebold recently posted "ML and Metrics VI: A Key Difference Between ML and TS Econometrics" on his blog. I am providing a shortened version of it, so all credit goes to him. (Emphasis in bold is mine.)

[S]tatistical machine learning (ML) and time series econometrics (TS) have lots in common. But there's also an interesting difference: ML's emphasis on flexible nonparametric modeling of conditional-mean nonlinearity doesn't play a big role in TS. <...>

[T]here's very little evidence of important conditional-mean nonlinearity in the covariance-stationary (de-trended, de-seasonalized) dynamics of most economic time series. <...> Indeed I can think of only one type of conditional-mean nonlinearity that has emerged as repeatedly important for (at least some) economic time series: Hamilton-style Markov-switching dynamics.

[Of course there's a non-linear elephant in the room: Engle-style GARCH-type dynamics. They're tremendously important in financial econometrics, and sometimes also in macro-econometrics, but they're about conditional variances, not conditional means.]

So there are basically only two important non-linear models in TS, and only one of them speaks to conditional-mean dynamics. And crucially, they're both very tightly parametric, closely tailored to specialized features of economic and financial data.

Thus the conclusion is:

ML emphasizes approximating non-linear conditional-mean functions in highly-flexible non-parametric fashion. That turns out to be doubly unnecessary in TS: There's just not much conditional-mean non-linearity to worry about, and when there occasionally is, it's typically of a highly-specialized nature best approximated in highly-specialized (tightly-parametric) fashion.

I recommend reading the whole original post here.

$\endgroup$
1
  • 1
    $\begingroup$ +1. I highly agree with this answer. Typical ML methods are characterised by nonparametric modelling and have very relaxed assumptions while ARMA models are "tightly-parametric". $\endgroup$
    – Digio
    Commented Aug 10, 2017 at 11:03
1
$\begingroup$

As said in other answers, ML methods required the i.i.d assumption and therefore are unsuitable to be used naively for time series. Not-naive ways have been suggested by DankMasterDan.

There is a further option, which is to use supervised predictors inside time-series forecasting. A good overview is found in the Sktime documentation: Forecasting, supervised regression, and pitfalls in confusing the two.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.