TensorFlow allows you to create MultiRNNCell composed sequentially of multiple simple cells (LSTM and GRU). I usually use same type of cell when creating MultiRNNCell but I was wondering if there could be some benefits in using both LSTM and GRU? Does anyone have some experience with it or theoretical insights?

  • 2
    $\begingroup$ Would be pretty interesting to see. I might be testing the combination of LSTM and NARX cells. $\endgroup$ – Thomas W May 11 '17 at 8:41

There are some thousands of variants of RNN cell(kernel) and both LSTM and GRU are for processing the input $x_i$ and the output of the previous state $s_{i-1}$ and producing the output and the current state. Even thought LSTM preceded GRU and GRU contains less computation, LSTM is just on a par with GRU in performance. So, I think stacking LSTM and GRU or any other cells might just interesting but would not make any big difference in improving the performance(compared to simply stacking either LSTM cells or GRU cells).

| cite | improve this answer | |
  • $\begingroup$ +1 for the last sentence: they really don't do things that are that different so there's not that much advantage you'd get from combining them. $\endgroup$ – Wayne Feb 8 '18 at 14:18
  • $\begingroup$ When you say that LSTM is on par with GRU, are you making the comparison on a per-unit basis or a per-parameter basis? $\endgroup$ – Sycorax Feb 8 '18 at 14:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.