Timeline for Why are the weights of RNN/LSTM networks shared across time?
Current License: CC BY-SA 3.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 27, 2022 at 16:25 | comment | added | eric_kernfeld | It would be really nice to see even a length-3 example where I give you an arbitrary RNN with different weights for 0 to 1 and 1 to 2 and you give me back an RNN that imitates it but using the same weights for 0 to 1 and 1 to 2. I suspect the imitator would need to be deeper or wider per-layer than the original, which makes this less of a free lunch and more of a tradeoff about adding complexity within vs between layers. | |
| Dec 27, 2022 at 16:13 | comment | added | eric_kernfeld | This is a fascinating comment, and it's nicely complementary to the other explanations on this thread. Do you have a reference, or could you explain how this works? | |
| Feb 4, 2017 at 20:21 | comment | added | Hossein | The RNNs are used to process sequence of data. One of their commonest types gets a sequence as input and produces another sequence as output (such as language translation systems). I say that an RNN model family M1 is more powerful than another RNN model family M2, if for a problem (such as mapping a set of input sequences to a set of output sequences) there is some model m1 in M1 where can solve this problem but there is no model in M2 where can solve that problem. | |
| Feb 4, 2017 at 17:27 | comment | added | whuber♦ | Could you elaborate on what you mean by "powerful"? The reference to Turing Machines suggests what you have in mind may be completely different than what is meant in statistics. | |
| Feb 4, 2017 at 16:59 | review | Late answers | |||
| Feb 4, 2017 at 17:27 | |||||
| Feb 4, 2017 at 16:33 | history | answered | Hossein | CC BY-SA 3.0 |