In Recurrent Neural Networks with connections from output units to the hidden units, we can use teacher forcing to make the training process faster by parallelization of learning in different time steps. In teacher forcing, we use the ground truth output in the current time step(available in the training data) to compute the system state in the next time steps. It is obviously faster than using the actual model output during training. But the question is that whether this is also more accurate? Maybe if we are not worried about training time, it is better to use the actual model output instead of ground truth outputs, since, when the model is deployed, the model output is ultimately used to produce the system state in next time steps.
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
I'll begin by saying I'm no expert but was thinking about this same question. A little googling led me to this page:
https://machinelearningmastery.com/teacher-forcing-for-recurrent-neural-networks/
and, in turn, this paper:
https://arxiv.org/abs/1610.09038
which as a paragraph addressing this to some degree in the introduction:
Unfortunately, [teacher forcing] can result in problems in generation as small prediction error compound in the conditioning context. This can lead to poor prediction performance as the RNN’s conditioning context (the sequence of previously generated samples) diverge from sequences seen during training
In addition, from the deeplearning.org book, 10, p.378:
The disadvantage of strict teacher forcing arises if the network is going to be later used in a closed-loop mode, with the network outputs (or samples from the output distribution) fed back as input. In this case, the fed-back inputs that the network sees during training could be quite different from the kind of inputs that it will see at test time.
I would imagine (again, not an expert) that it is fairly problem-dependent, but that the main gain of teacher forcing is in the computational training and simplifying the loss landscape (i.e. since the whole sequence will contribute to the gradient of the parameters, for long sequences backpropagation through time may make it difficult for the optimiser to converge even if it has a lot of computational time.)
$\begingroup$
$\endgroup$
You might want to read this paper: https://arxiv.org/abs/1905.10617
"Exposure bias has been regarded as a central problem for auto-regressive language models (LM). It claims that teacher forcing would cause the test-time generation to be incrementally distorted due to the training-generation discrepancy. Although a lot of algorithms have been proposed to avoid teacher forcing and therefore alleviate exposure bias, there is little work showing how serious the exposure bias problem actually is. In this work, we focus on the task of open-ended language generation, propose metrics to quantify the impact of exposure bias in the aspects of quality, diversity, and consistency. Our key intuition is that if we feed ground-truth data prefixes (instead of prefixes generated by the model itself) into the model and ask it to continue the generation, the performance should become much better because the training-generation discrepancy in the prefix is removed. Both automatic and human evaluations are conducted in our experiments. On the contrary to the popular belief in exposure bias, we find that the the distortion induced by the prefix discrepancy is limited, and does not seem to be incremental during the generation. Moreover, our analysis reveals an interesting self-recovery ability of the LM, which we hypothesize to be countering the harmful effects from exposure bias"
