What is the most efficient method to handle long time sequences (LSTM)? I am using LSTM and I have several long time sequences of varying length. Most of them are about 6,000-7,000 timesteps on average, but several are around 40,000 long. I am not sure which of this would be the best choice:


*

*Leave them as they are. Pad the short sequences and use truncated backpropagation (but would it work, since the sequences have so many zeros?).

*Split only the biggest sequences so that all the sequences are around 6,000-7,000 sequences. Pad some of them if needed. (I think this is the best option but I am not sure.)

*Split all of them to the smaller sequences. (I read this question: What is a feasible sequence length for an RNN to model? and according to it it seems that there is no need to do so, and it's just better to use truncated backpropagation on the longer sequences. Am I right about it?)
 A: If  the training time is an issue, a typical strategy is to create mini-batches in which sequences have approximately the same length.
E.g., from {1}:

In order to minimize the zero padding for each batch, during the process of loading the data and
  creating the mini-batches, the samples are sorted with respect to the number of frames. By doing this,
  each mini-batch has a minimum variation in number of time steps and the network does not have to
  process as much silence corresponding to the padded zeros. This technique is called bucketing.

See {2} for some comparison shuffle vs. create mini-batches in which sequences have approximately the same length.

References:


*

*{1} Ramos, M.V., 2016. Voice Conversion with Deep Learning. Tecnico Lisboa Master’s thesis.

*{2} Morishita, Makoto, Yusuke Oda, Graham Neubig, Koichiro Yoshino, Katsuhito Sudoh, and Satoshi Nakamura. "An empirical study of mini-batch creation strategies for neural machine translation." arXiv preprint arXiv:1706.05765 (2017). https://arxiv.org/pdf/1706.05765.pdf
