1
$\begingroup$

I've got a question about an LSTM neural net fitting a random walk. I've made the LSTM [network shape: 1, 50, 100, 200, 50, 1] and out of interest made a completely random walk (by using a normal distribution in Excel to create the walk). I've chosen to feed the network sequences of 50 prior timesteps in an overlapping window that progresses by 1 each time (with the Y component being the next timestep).

I've split the data 90/10 train/test and run it on the test data (so that for each timestep it predicts the next timestep using the 50 prior timestep window, then updates the window with the next timestep and predicts the one after that...etc...) but after training it and running it across the test set I get the below:

LSTM Random Walk

Now this has very much confused me as it's clearly fitted extremely well, which theoretically surely shouldn't be the case if the underlying data is a random walk which, by it's nature, has no sort of predictable patterns?

Does anybody know why it's managed to fit like this (my only trail of thought is that somehow it's figured out the sequence of the pseudo-random number generator in excel through the training... but I feel that seems unlikely, no?).

$\endgroup$
  • 1
    $\begingroup$ This will certainly be an instance of overfitting. Your out of sample forecasting ability will be terrible. $\endgroup$ – gung Oct 4 '16 at 16:08
  • $\begingroup$ But surely the fact that I've separated training and testing sets means it can't overfit, as it would have never seen the test data before, and only trained on the training data? $\endgroup$ – Jakob Oct 4 '16 at 16:18
  • 1
    $\begingroup$ This is a time series. What does it mean to have separate training & testing sets? $\endgroup$ – gung Oct 4 '16 at 16:24
  • $\begingroup$ @Jakob Why don't you do a little more coding (should be easy to do). Let's say this is stock market and you could buy (or sell) the random walk at time t if your prediction tells you it's going up (down). Then sell (buy back) the RW at t+1. For every trade, save the profit/loss. Do this over a N steps. See what happen. $\endgroup$ – horaceT Oct 4 '16 at 17:16
  • $\begingroup$ Suppose that you used only data up until t=400. How well does your model predict t=401, 402, ... 500? $\endgroup$ – Sycorax Oct 6 '16 at 0:32
1
$\begingroup$

With a random walk, the best predictor is simply the previous observation. You should actually attempt to forecast first order differences. For a random walk, your prediction for the first order difference will be statistically close to zero.

$\endgroup$
  • 1
    $\begingroup$ Thanks @SJB... Could you clarify what you mean by First Order Difference... As in the period change (e.g. natural log of each series to it;s t-1??) $\endgroup$ – Jakob Nov 16 '16 at 20:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.