Timeline for Teacher Forcing in RNNs
Current License: CC BY-SA 4.0
3 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 9, 2021 at 11:11 | comment | added | boomkin | Yes, I mean that. There are good reasons to use teacher forcing, and I think in generic RNN training in PyTorch, it would be assumed that you are using teacher forcing because it is just faster. One way to look at is that you could have measurement error in your data, and the RNN functions like a filter trying to correct it. So if you want to introduce robustness to thee kind of measurement errors, it makes sense to sample both from the predicted and the observed samples. It is highly domain dependent whether it works but it can be treated as an additional hyperparameter. | |
| Jan 8, 2021 at 23:17 | comment | added | Eisen | Hi! Thank you for your answer. What do you mean by "it is better to condition on its own faulty...better chance of mitigating its own mistakes."? Do you mean that we should not use teacher forcing 100% of the time so that we will have instances where mistakes are more likely, which is better for learning? | |
| Jan 8, 2021 at 22:04 | history | answered | boomkin | CC BY-SA 4.0 |