I've got a question about an LSTM neural net fitting a random walk. I've made the LSTM [network shape: 1, 50, 100, 200, 50, 1] and out of interest made a completely random walk (by using a normal distribution in Excel to create the walk). I've chosen to feed the network sequences of 50 prior timesteps in an overlapping window that progresses by 1 each time (with the Y component being the next timestep).
I've split the data 90/10 train/test and run it on the test data (so that for each timestep it predicts the next timestep using the 50 prior timestep window, then updates the window with the next timestep and predicts the one after that...etc...) but after training it and running it across the test set I get the below:
Now this has very much confused me as it's clearly fitted extremely well, which theoretically surely shouldn't be the case if the underlying data is a random walk which, by it's nature, has no sort of predictable patterns?
Does anybody know why it's managed to fit like this (my only trail of thought is that somehow it's figured out the sequence of the pseudo-random number generator in excel through the training... but I feel that seems unlikely, no?).