Can anyone recommend any machine learning techniques for time series estimation?

I have a series of times $t_{1}...t_{n}$, each having a set of associated features $f_{1}...f_{m}$, and a value $x$.

I want to estimate the value of $x$ for the next time in the series given a set of corresponding features $f_{1}...f_{n}$.



Just to add clarification, this is for price forecasting, where x is a price and the features will be extracted from news articles.


An ARMAX model might be a good place to start.

  • 1
    $\begingroup$ Quite correct ! Care should also be taken to test the Gaussian Assumptions and to introduce needed remedies, if necessary. $\endgroup$ – IrishStat Jul 13 '11 at 14:21
  • $\begingroup$ @Zach thanks. Can anyone recommend any implementations, in python, R, etc? $\endgroup$ – S0rin Jul 16 '11 at 7:37
  • 1
    $\begingroup$ @S0rin: I would recommend starting with the arima function in R. Use the xreg argument for your regressors. You may also find the auto.arima function in the forecast package useful (this function also has the xreg argument). $\endgroup$ – Zach Jul 17 '11 at 2:09
  • $\begingroup$ @IrishStat (and Zach), would this model be suitable if the set of features is also a time series? $\endgroup$ – S0rin Aug 16 '11 at 17:01
  • $\begingroup$ @Moominpappa: Yes. ARMAX models are often used with independent variables that vary over time. $\endgroup$ – Zach Aug 16 '11 at 18:16

Recurrent neural networks:

  • no assumptions on the distributions of $f_i$,
  • distribution of $x_t$ can be modelled via an adequate loss function (sum of squares for Gaussian, sum of differences for Laplace, cross entropy, kulback leibler divergences, ...)
  • rather difficult to implement (need advanced techniques such as Hessian free optimization or Long short-term memory to work well).

PyBrain has a LSTM implementation.


Critically, what's your data frequency? Is this high or low frequency news? Also critically, you want to forecast returns, not prices.

Your question is sufficiently broad that every supervised learning/regression technique can be listed legitimately.

For example, your news could be high frequency news, meaning the response is basically an inhomogeneous time series and a discrete process. Whereas if it is monthly data it is much more Gaussian but is also much more efficiently priced and you have no sample size to test your model's ability to generalise. Data frequency, the market's liquidity, microstructure and other domain specific issues will completely change the statistical model chosen.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.