I just started learning deep learning a few weeks ago. Currently I am working on a deep learning project that uses tensorflow to predict stock prices. I think I currently lack a bigger picture of how DP can be applied to this type of problem due to the insufficent data set and features. I will post my thoughts and what I've done and would like to hear people's opionions on this.

Here is what I've done so far.

Let's say I pick one stock and its history prices of past 30 years, and price is given in a montly basis. And I use that as my Y.

For X, I use features like company revenues, profits, growth rate, etc for that company as well as interest rate, inflation rate. Also that data is spread in a monthly basis.

So the total sample size is about ~550 (80% used as training set and 20% as testing set) and currently I am just using various shallow layers (1-3) and with various hidden units in each layer (2-30) along with combinations of different learning rate, optimizers, batch sizes etc. So far I couldn't get any good results.

Am I doing something completely off the chart here? I would like to make sure I am at least on the right track.

  • 1
    $\begingroup$ Relationship of stock price to measurable features is complex, you have a lot of possible external factors. The price you see is being affected by many active agents who all benefit from accurately predicting it so they can buy and sell at the right time - whilst the impact of their choices following predictions also affects the price, causing feedback. It is generally considered a hard problem, and supervised learning techniques with easily-available public data will not get you very far $\endgroup$ – Neil Slater Sep 10 '17 at 20:53
  • 2
    $\begingroup$ Could you try making your question more focused? What exactly is the problem in here? Usually "what are your thoughts about..." kind of questions would be off-topic in here, so please make it more precise. $\endgroup$ – Tim Sep 10 '17 at 20:53
  • $\begingroup$ This is an open question that I just want to hear ppl's thoughts on this. Thanks @NeilSlater $\endgroup$ – Dude from SF Sep 10 '17 at 20:58
  • $\begingroup$ I think my goal here is not to make profits based on the prices. I only would like to build a model that predicts the past price. Say I want to use all the data from 2000-2010 to predict the price of certain stock in 2011 with all the features given in that year. $\endgroup$ – Dude from SF Sep 10 '17 at 21:00
  • 2
    $\begingroup$ @HaoZeng deep learning is a big buzzword nowdays but this doesn't mean that it is a one-size-fit-all tool. Many popular machine learning algorithms fail miserably as compared to old-school time-series models for the time-series data. The more appropriate way to go is to start with a problem and then look for a way of solving it, rather then starting with a solution and then wondering if you can solve your problem with it... $\endgroup$ – Tim Sep 10 '17 at 21:11

Pros: If you can actually do it accurately, fast and secretly, for as long as the market assumptions stay stationary, you will get rich very quickly with relatively little labour input.

Cons: Practically impossible to do at any retail level. Market assumptions change quickly over time so models can quickly go from good to useless. Highly competitive field so any relevant insights made will be made in the private space and will practically never be made public. Accurate data for training can be expensive to obtain, public data will pretty much be useless for anything serious. Literally thousands of covariates used in accurate models and need to be trained for efficient models. Generally noisy data with a lot of feedback, prices themselves by definition are essentially just agents decisions on prices being too high/too low, derived from models causing price action outside of predictors themselves creating a loop and breaking model covariate assumptions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.