In order to train an LSTM/RNN in batches, the sequences in the batch need to be the same length. From my understanding, this can be done by either truncating longer sequences, padding shorter ones, or some combination of the two.
It seems advantageous to batch sequences with similar original lengths, so that there is minimal need for padding/truncation. Two questions:
1) Is this true? Should I group similar-length sequences into the same batch for training? It seems it would be more efficient, but also it seems it may introduce some biases since similar-length sequences may share other similarities (for example, in classification, shorter sequences may be more likely to be a certain class).
2) What is the best way to select, from a corpus of N sequences of varying lengths, a subset of sequences which are "similar" in size? I was thinking of ordering the sequences by lengths, picking one randomly, and sampling its neighbors somehow.