6
$\begingroup$

So I have been trying to train an LSTM to predict the values of a certain stock. The error was pretty low, so I decided to create a graph out of the test set. It looked like this:

Red: actual, Black: my predition, Blue: input to get prediction

So i'm training the network with datsets like: in: xt-1, out: xt. But when ignoring the fact that the black line is much lower than the red line, you'll see that the network is actually mimicking the input to stay as close to the actual prediction.

So after doing some googling, I found out that this is a common 'trap:

I am sure there is one step lag between the actual time series and the predicted time series, this is the most seen "trap" if you do time series prediction like this, in which the NN will always mimic previous input of time series.

But are there things I do to avoid this? In the thread I linked some solutions are pointed out, but are there any better, generic solutions?


I have created a JSFiddle with the training of the neural network and the chart. View it here (open console before opening). Feel free to tweak around with the options to see if you get something working...

$\endgroup$
4
+25
$\begingroup$

This is a very common problem. Here is an article with a good explanation. People have been trying to do this for a very long time, and it is safe to say there is no such thing as model that can tell you what the price of a financial instrument will be in the future with good accuracy. Some people believe that there isn't any way to predict the future prices of the market any better than flipping a coin (Random walk theory, Here is a good explanation of it). This is totally not true though. The movement of markets is almost a random walk, but not 100% random.

This does not mean that machine learning has no use for trading however. I won't give away too much, and anyone who has figured out how to use it to their advantage won't mention their details either, but you have to get outside the box a bit. Putting in price history and getting a future price is way too good to ever be true. A better approach is using multiple different networks for several different easier problems, and using what they come up with and your own knowledge to screen for a good stock to buy. If you really want to use machine learning for trading, you are going to have to be very dedicated to it and invest a huge amount of time. You not only need to master the machine learning side of it, but you will also have to master the market.

$\endgroup$
1
  • $\begingroup$ Very interesting article. Helped me a lot already. Got exactly the same results as the author had. $\endgroup$ Aug 1 '17 at 9:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.