6
$\begingroup$

I have a set of sequences. Each sequence is the form $\{(s_1,l_1),(s_2,l_2) \ldots\}$ where $s_i$'s are real valued numbers and $l_i$s are labels from a fixed alphabet. It is important to note that the sequences may be of different lengths. Pictorially,

enter image description here

I would like to predict the labels corresponding to a test sequence. With reference to the picture below, I would like to predict the red $l_i$s.

enter image description here

How do I use the LSTM framework of Keras to solve this ? Examples or reference links would also be welcome.

|cite|improve this question
$\endgroup$
  • $\begingroup$ have you looked at github.com/fchollet/keras/tree/master/examples ? $\endgroup$ – ruoho ruotsi Feb 10 '16 at 5:14
  • $\begingroup$ @ruohoruotsi It looks like keras cannot handle variable length sequences. The best I could think of is to batch up data into sequences of equal length. $\endgroup$ – curryage Feb 10 '16 at 9:41
2
$\begingroup$

I see there was an issue filed last year about this. The author recommends zero-padding or batches of size 1:

Zero-padding

X = keras.preprocessing.sequence.pad_sequences(sequences, maxlen=100)
model.fit(X, y, batch_size=32, nb_epoch=10)

Batches of size 1

for seq, label in zip(sequences, y):
   model.train(np.array([seq]), [label])
|cite|improve this answer
$\endgroup$
  • $\begingroup$ For zero-padding, do I also need to pad y? $\endgroup$ – Munichong Aug 28 '16 at 18:19
  • 1
    $\begingroup$ If the Ys are sequences, they may need to have a consistent shape (but not necessarily the same shape as the Xs), in this case, you will need to pad them. However, if the Ys are single labels (for say classification) you don't need to/cannot pad them. Have a look here: github.com/fchollet/keras/issues/395 stackoverflow.com/questions/37307421/… $\endgroup$ – ruoho ruotsi Aug 29 '16 at 4:27
  • $\begingroup$ Thx. How do I decide if I should use 'pre' or 'post'? I am doing sth like time series prediction. Each training case has different length. Given previous several X1...(t-1) and Y1...(t-1) and current Xt, the goal is to predict Yt. $\endgroup$ – Munichong Aug 29 '16 at 11:26
  • 1
    $\begingroup$ Pre or Post padding refers to putting the zeros to the left or right of the sequence (left vs. right-padding) ... depending on your application, one may make more sense that the other. Given a sequence (s1,...s7) s7 is the last time step, s1 the earliest. Many text/NLP sequences application left-pad to put the zeros before the oldest part of the sequence, like (0,0,0,s1..s7) Contrast this with post (right-padding (s1...s7,0,0,0) which may disrupt the effectiveness of the LSTM to learn that s7 is the most recent item. $\endgroup$ – ruoho ruotsi Aug 29 '16 at 18:24
  • $\begingroup$ Thanks a lot. Actually, I post a question about my RNN project. I am not sure if you have any idea. Could you please kindly help if possible? stats.stackexchange.com/questions/232519/… $\endgroup$ – Munichong Aug 30 '16 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.