# Using Keras LSTM RNN for variable length sequence prediction

I have a set of sequences. Each sequence is the form $\{(s_1,l_1),(s_2,l_2) \ldots\}$ where $s_i$'s are real valued numbers and $l_i$s are labels from a fixed alphabet. It is important to note that the sequences may be of different lengths. Pictorially,

I would like to predict the labels corresponding to a test sequence. With reference to the picture below, I would like to predict the red $l_i$s.

How do I use the LSTM framework of Keras to solve this ? Examples or reference links would also be welcome.

• have you looked at github.com/fchollet/keras/tree/master/examples ? – ruoho ruotsi Feb 10 '16 at 5:14
• @ruohoruotsi It looks like keras cannot handle variable length sequences. The best I could think of is to batch up data into sequences of equal length. – curryage Feb 10 '16 at 9:41

X = keras.preprocessing.sequence.pad_sequences(sequences, maxlen=100)
model.fit(X, y, batch_size=32, nb_epoch=10)


Batches of size 1

for seq, label in zip(sequences, y):
model.train(np.array([seq]), [label])

• For zero-padding, do I also need to pad y? – Munichong Aug 28 '16 at 18:19
• If the Ys are sequences, they may need to have a consistent shape (but not necessarily the same shape as the Xs), in this case, you will need to pad them. However, if the Ys are single labels (for say classification) you don't need to/cannot pad them. Have a look here: github.com/fchollet/keras/issues/395 stackoverflow.com/questions/37307421/… – ruoho ruotsi Aug 29 '16 at 4:27
• Thx. How do I decide if I should use 'pre' or 'post'? I am doing sth like time series prediction. Each training case has different length. Given previous several X1...(t-1) and Y1...(t-1) and current Xt, the goal is to predict Yt. – Munichong Aug 29 '16 at 11:26
• Pre or Post padding refers to putting the zeros to the left or right of the sequence (left vs. right-padding) ... depending on your application, one may make more sense that the other. Given a sequence (s1,...s7) s7 is the last time step, s1 the earliest. Many text/NLP sequences application left-pad to put the zeros before the oldest part of the sequence, like (0,0,0,s1..s7) Contrast this with post (right-padding (s1...s7,0,0,0) which may disrupt the effectiveness of the LSTM to learn that s7 is the most recent item. – ruoho ruotsi Aug 29 '16 at 18:24
• Thanks a lot. Actually, I post a question about my RNN project. I am not sure if you have any idea. Could you please kindly help if possible? stats.stackexchange.com/questions/232519/… – Munichong Aug 30 '16 at 16:43