I've been searching for a while now to find the precise way to feed a Recurrent Neural Network (RNN, LSTM, GRU, ESN, Etc) with time series data with no real success.
Here is a question that was close, but the answers aren't very clear: Proper way of using recurrent neural network for time series analysis
I'm not looking for a breakdown of how the networks work, but rather how to structure the input/output vectors for optimal results.
So, let's say I'm working with a growing sinusoid. Here is my data set with 50 values:
{ 40, 43.545, 47.214, 50.518, 52.959, 54.102, 53.639, 51.449, 47.621, 42.47, 36.506, 30.392, 24.864, 20.646, 18.356, 18.424, 21.022, 26.027, 33.015, 41.295, 49.977, 58.067, 64.579, 68.656, 69.681, 67.358, 61.772, 53.394, 43.044, 31.816, 20.959, 11.741, 5.299, 2.5, 3.833, 9.327, 18.537, 30.57, 44.172, 57.857, 70.071, 79.368, 84.577, 84.954, 80.276, 70.889, 57.687, 42.03, 25.605, 10.242 };
I am very familiar with the sliding time window approach that works well with Feed forward networks (FFN). Normalization aside, I would do something like this:
Train with 49 of the 50 values and validate with the last
3 inputs, 1 output - Sliding Time Window
Train:
Input Output
-------------------------------------
[1] 40, 43.545, 47.214 -> 50.518
[2] 43.545, 47.214, 50.518 -> 52.959
...
[49] 70.889, 57.687, 42.03 -> 25.605
Validate:
Input Output
-------------------------------------
[50] 57.687, 42.03, 25.605 -> 10.242
And this works very well with FFN (And with RNN also), but I'm led to believe RNN shouldn't need to be setup that way. Rather they work very well with time series data and all I should need need to do is have the input be the previous output. Something like this:
Train with 49 of the 50 values and validate with the last
1 input, 1 output
Train:
Input Output
-------------------------------------
[1] 40 -> 43.545
[2] 43.545 -> 47.214
...
[49] 42.03 -> 25.605
Validate:
Input Output
-------------------------------------
[50] 25.605 -> 10.242
But this has produced horrible results with my RNNs. I can't seem to get a handle on how to feed time series data. I've tried the input being the index, such as:
Input Output
-------------------------------------
[1] 1 -> 40
[2] 2 -> 43.545
[3] 3 -> 47.214
I've even tried combinations of this and the previous setup and nothing is working well.
Could someone please show me exactly how the input and output vectors should be for this scenario?
timestep = 3
. If you want memory to pass through each iteration you need your RNN to be stateful. $\endgroup$ – Akylas Stratigakos Dec 13 '18 at 10:56