Best way to mini-batch similar lengths sequences in a corpus for RNN training? In order to train an LSTM/RNN in batches, the sequences in the batch need to be the same length. From my understanding, this can be done by either truncating longer sequences, padding shorter ones, or some combination of the two. 
It seems advantageous to batch sequences with similar original lengths, so that there is minimal need for padding/truncation. Two questions:
1) Is this true? Should I group similar-length sequences into the same batch for training? It seems it would be more efficient, but also it seems it may introduce some biases since similar-length sequences may share other similarities (for example, in classification, shorter sequences may be more likely to be a certain class).
2) What is the best way to select, from a corpus of N sequences of varying lengths, a subset of sequences which are "similar" in size?  I was thinking of ordering the sequences by lengths, picking one randomly, and sampling its neighbors somehow. 
 A: 
Is this true? Should I group similar-length sequences into the same batch for training? It seems it would be more efficient, but also it seems it may introduce some biases since similar-length sequences may share other similarities (for example, in classification, shorter sequences may be more likely to be a certain class).

This is true. Grouping similar-length sequences into the same batch will speed up training (as well as testing), while introducing some bias.

What is the best way to select, from a corpus of N sequences of varying lengths, a subset of sequences which are "similar" in size? I was thinking of ordering the sequences by lengths, picking one randomly, and sampling its neighbors somehow.

That sounds like a good trade-off between shuffling batches and speed. I'm not aware of any standard solution to this trade-off: from what I have read so far, it's an empirical decision.
FYI:


*

*Neural networks: why do we randomize the training set?

*When the data set size is not a multiple of the mini-batch size, should the last mini-batch be smaller, or contain samples from other batches?
