Are RNNs in Keras dynamic length?

Question

Sorry if this seems like a basic question.

From my understanding, one of the advantages of sequence models like RNNs is that they can handle variable length input sequences. For example, if I'm doing sentiment classification on tweets, I can scale the number of RNN units to match the number of words in my input.

But a lot of implementations of NLP projects that I have seen seem to fix the input length to be fed into Keras. They find the longest length input and pad the others to match.

Is this required theoretically? I would assume not. If it is not then is it to fix the computation graph a priori? Or can keras handle variable length inputs? I am a little confused. Any help appreciated. Thanks in. advance.

Answer 1

This is perfectly reasonable question, which is actually related to purely technical aspect of deep learning frameworks. I assume you know how Backpropagation Through Time (BPTT) works since you're asking about RNNs.

TL;DR - Keras was built to support frameworks like Theano and Tensorflow which supported only static computation graphs.

From my understanding, one of the advantages of sequence models like RNNs is that they can handle variable length input sequences.

This is true, but in practice

• you wouldn't actually use inputs of arbitrary lengths, since standard RNN vanishing/exploding gradient problems
• you don't actually need arbitrary length because your dataset will have a longest input
• you can easily adapt shorter sequences to longer - by padding them

Why would you want to adapt all sequences to have same length? That's basically because it will allow you to do batching.

Why would you want to keep the size of tensors constant when running your data by batches? This is no longer required (for example in PyTorch and MXNet you can unroll your network dynamically), but older frameworks like Theano and Tensorflow (until recently) required you to write static computation graphs. Basically this means they required you to know the shape of data before you feed it into NN - creating a single computation graph and then rerunning it was significantly faster than creating a graph each time you execute it.