0
$\begingroup$

I have timeseries data with 1 minute cadence with 4 features, and I want to try to predict the time-evolution of 2 of these features using a RNN using LSTMs in Keras. My aim is to predict the e.g. next 10 minutes of data for these two features, based on the last hour of data of the 4 features.

I have previously constructed a RNN which can predict a single timestep for each 60 minute data input, using keras (many-to-one). Since I only predict a single timestep (minute), I need to predict all 4 features, to be able to use this model to predict up to 10 timesteps (minutes) iteratively. The model looks like this:

from keras.models import Sequential
from keras.layers import Dense, LSTM, Dropout, Activation, BatchNormalization

model = Sequential()
model.add(LSTM(32, return_sequences=True, input_shape=(shape,4)))
# ... more hidden LSTM layers
model.add(LSTM(32))
model.add(Dense(4))

My input data are timeseries of the features, data. I use MinMaxScaler in scikit-learn to normalize the features to (0,1). I then use TimeseriesGenerator from keras to generate the training data. I use a length of 60 to provide the RNN with 60 timesteps of data in the input.

from keras.preprocessing.sequence import TimeseriesGenerator

# data.shape is (n,4), n timesteps
tsgen = TimeseriesGenerator(data, data, length=60, batch_size=240)

I then fit the model, with checkpointing:

mcp = ModelCheckpoint("rnn_{epoch:03d}_{loss:.6f}.h5")
model.fit_generator(tsgen, epochs=30, callbacks=[mcp])

I would now like to a prediction "many-to-many", so that I do not need to predict all 4 features, but rather predict the next 10 timesteps of the 2 sought-after features.

I have looked at the TimeDistributed layer for keras, but according to the documentation that will only allow me to predict 60 values (i.e. the length of my input timestep shape). How do I use that to predict only the next 10 minutes of data, based on the last 60 minutes? How would I write that in keras? Do I need to use my own TimeseriesGenerator to account for the target vector? For example, is it sufficient to fill the last 50 minutes of the target vector with zeros, and just use the TimeDistributed layer?

$\endgroup$
0
$\begingroup$

If you're still interested in this, I would check out this thread: https://github.com/keras-team/keras/issues/6063. A sample model might look like this:

model = Sequential()  
model.add(LSTM(input_dim=1, output_dim=hidden_neurons, return_sequences=False))  
model.add(RepeatVector(X)) #X = number of time steps you want to forecast
model.add(LSTM(output_dim=hidden_neurons, return_sequences=True))  
model.add(TimeDistributed(Dense(1)))
model.add(Activation('linear'))   
model.compile(loss='mean_squared_error', optimizer='rmsprop', metrics=['accuracy']) 

I am also pretty interested in doing something similar and might end up coming back to it because it looks promising. I hope this helps!

$\endgroup$
5
  • $\begingroup$ Although implementation is often mixed with substantive content in questions, we are supposed to be a site for providing information about statistics, machine learning, etc., not code. It can be good to provide code as well, but please elaborate your substantive answer in text for people who don't read this language well enough to recognize & extract the answer from the code. $\endgroup$ – gung - Reinstate Monica Oct 4 '19 at 1:26
  • $\begingroup$ Thank you for the answer, I have yet to understand how and why this is an implementation of "many-to-many". Isn't this example still "many-to-one", i.e. it predicts a single value? Though I have not invested more time into this, since this was just a quick-shot pet project, to do a proof-of-concept. $\endgroup$ – Christoph Terasa Oct 4 '19 at 9:36
  • $\begingroup$ @gung it's clear the author understands code, which is why I included it in the answer. I haven't provided a further explanation because the link I included provides more information than I can provide--hence my linking to it. If code is the limiting factor for other readers, then I don't believe this question is for them. I wholeheartedly understand where your comment is coming from, but I believe it serves more of a purpose for code-heavy answers to questions where code is not originally mentioned or isn't critical to the question. That is not the case here. $\endgroup$ – eorland Oct 4 '19 at 21:22
  • $\begingroup$ @ChristophTerasa My elementary understanding of the TimeDistributed layer is that it returns a sequence of values. This would make it 'many to many', although if you consider the sequence a single output, then it would be 'many to one', but I don't think that is correct. Like for most Machine Learning things, there is a Machine Learning Mastery article about it: machinelearningmastery.com/…. This helps clarify that the TimeDistributed layer can return sequences. $\endgroup$ – eorland Oct 4 '19 at 21:26
  • $\begingroup$ Last thing! Still sifting through the noise. I think this article is a great resource: machinelearningmastery.com/… $\endgroup$ – eorland Oct 8 '19 at 4:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.