Timeline for How to choose between plain vanilla RNN and LSTM RNN when modelling a time series?
Current License: CC BY-SA 3.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 1, 2017 at 15:14 | vote | accept | mickkk | ||
| Nov 9, 2016 at 17:16 | history | edited | Franck Dernoncourt | CC BY-SA 3.0 |
add LSTM: A Search Space Odyssey
|
| Aug 8, 2016 at 21:15 | comment | added | Munichong | Hi, @FranckDernoncourt. If we have short sequence (usually 3-4 status in each sequence), does it overkill to use LSTM than RNN? (based on my understanding, LSTM is especially good for long sequence) | |
| Jul 31, 2016 at 23:36 | history | edited | Franck Dernoncourt | CC BY-SA 3.0 |
added 321 characters in body
|
| Jul 31, 2016 at 23:36 | comment | added | Franck Dernoncourt | @horaceT Le, Quoc V., Navdeep Jaitly, and Geoffrey E. Hinton. "A simple way to initialize recurrent networks of rectified linear units." arXiv preprint arXiv:1504.00941 (2015). shows that RNNs can sometime have performances similar to LSTMs when the identity matrix is used to initialize the recurrent weight matrix. | |
| Jul 28, 2016 at 20:50 | comment | added | horaceT | @Franck I read Zaremba a while back, it shows there are other variants of LSTM that performs better. Thanks for pointing to this paper. | |
| Jul 28, 2016 at 20:45 | comment | added | Franck Dernoncourt | @horaceT nothing off the top of my head, but since LSTM has more parameters I'd guess there could be some corner cases where RNNs perform better. In practice, personally I directly try LSTM/GRU. | |
| Jul 28, 2016 at 20:44 | comment | added | horaceT | @Frank Can you cite a paper where plain RNN outperforms LSTM? | |
| Jul 28, 2016 at 20:44 | comment | added | mickkk | Empirically and/or "brute force testing" methods seem very common as I dive deeper in machine learning :). But that makes sense. Thanks. | |
| Jul 28, 2016 at 20:38 | history | answered | Franck Dernoncourt | CC BY-SA 3.0 |