What machine learning algorithm can be used to predict the stock market? Alternatively, to predict foreign exchange markets. I know this can get pretty complicated, so as an introduction, I'm looking for a simple prediction algorithm that has some accuracy.
(It's for a M.Sc. university project that lasts four months)
I've read that a multi-layer neural network might be useful. Any thoughts on that? In addition, semantic analysis of social media may provide insight into market behavior which influences the stock market. However, semantic analysis is a bit outside the scope of the project at the moment.
 A: You could try the auto.arima and ets functions in R.  You might also have some success with the rugarch package, but there's no existing functions for automated parameters selection.  Maybe you could get parameters for the mean model from auto.arima, then pass them to rugarch and add garch(1,1)?
There's all sorts of blogs out there that claim some success doing this.  Here's a system using an arima model (and later a garch model) and system using an SVM model. You'll find a lot of good info on FOSS trading, particularly if you start reading the blogs on his blogroll.
Whatever model you use, be sure to cross-validate and benchmark!  I'd be very surprised if you found an arima, ets, or even garch model that could consistantly beat a naive model out-of-sample. Examples of time series cross-validation can be found here and here.  Keep in mind that what you REALLY want to forecast is returns, not prices.
A: I know of one machine learning approach which is currently in use by at least one hedge fund. numer.ai is using an ensemble of user-provided machine learning algorithms to direct the actions of the fund.
In other words:
A hedge fund provides open access to an encrypted version of data on a couple of hundred investment vehicles, most likely stocks. Thousands of data scientists and the like train all sorts of machine learning algorithms against that data and upload the results to a scoreboard. The highest scorers get a small amount of money depending on the accuracy of their results and how long their result has been available online.
The best predictions are supposedly made by ensembles of algorithms.
So you have a lot of scientists providing trained guesses, some of which are themselves ensembles of guesses and the hedge fund uses the ensemble of all provided guesses to direct their investments.
This rather interesting hedge fund's results taught me two things:


*

*Ensembles are often viewed as a good way of making predictions on the stock market.

*Good predictions require more ensembles than I'm willing to build myself...


If you want to have a go, visit: https://numer.ai/
No, I'm NOT affiliated with them, I'd most likely not spend my days online were I connected to a hedge fund that employs thousands of people, but paying only those that provide measurable results :)
The numer.ai community has a forum where they discuss their approach so you CAN learn from others who are trying to do the same.
Personally I think anyone with a good algorithm is going to keep it very, very secret.
A: As babelproofreader mentioned, those that have a successful algorithm tend to be very secretive about it.  Thus it's unlikely that any widely available algorithm is going to be very useful out of the box unless you are doing something clever with it (at which point it sort of stops being widely available since you are adding to it).
That said, learning about autoregressive integerated moving average (ARIMA) models might be a useful start for forecasting time-series data.  Don't expect better than random results though.
A: I think for your purposes, you should pick a machine learning algorithm you find interesting and try it.
Regarding Efficient Market Theory, the markets are not efficient, in any time scale. Also, some people (both in academia and real-life quants) are motivated by the intellectual challenge, not just to get-rich-quick, and they do publish interesting results (and I count a failed result as an interesting one). But treat everything you read with a pinch of salt; if the results are really good, perhaps their scientific method isn't.
Data Mining With R might be a useful book for you; it is pricey, so try and find it in your university library. Chapter 2 covers just what you want to do, and he gets best results with a neural net. But be warned that he gets poor results, and spends a lot of CPU time to get them. The Amazon reviews point out the book costs $20 more because that chapter mentions the word finance; when reading it I got the impression the publisher had pushed him to write it. He's done his homework, read the docs, perused the right mailing lists, but his heart was not in it. I got some useful R knowledge from it, but won't be beating the market with it :-)
A: To my mind, any run-of-the-mill strong AI that could do all of the following might easily produce a statistically significant prediction:


*

*Gather and understand rumours

*Access and interpret all government knowledge

*Do so in every relevant country

*Make relevant predictions about:


*

*Weather conditions

*Terrorist activity

*Thoughts and feelings of individuals

*Everything else that affects trade
Statistical analysis is the least of your worries, really.
A: You should try GMDH-type neural networks.
I know that some successful commercial packages for stock market prediction are using it, but mention it only in the depths of the documentation.
In a nutshell it is a multilayered iterative neural network, so you are on the right way.
A: I think hidden markov models are popular in stock market. The most important thing to keep in mind is that you want an algorithm that preserves the temporal aspect of your data.
