Under periodic BPTT, is softmax evaluated only at the end of the period?

Question

Suppose I have a continuous sequence $X$ of words and I wish to train a RNN language model. According to [1], I would split $X$ into subsequences $X^{1..|X|/k_1}$ $k_1$ sized subsequences ($k_1$ is our period), then for each subsequence $X^i$, propagate $X^i_1$ through the network, then $X^i_2$, ... $X^i_{k_1}$, storing the hidden state of the network for the last $k_2$ of these forward propagations. Finally, the softmax is evaluated only for the $X^i_{k_1}$ word, ie: how well did the the network predict this last word. This error is then backpropagated through time for $k_2$ time steps, using the hidden state for these time steps that we stored previously to calculate the gradients.

My question is: I feel it's a bit strange to only evaluate the softmax at the end of the $k_1$ period. How does doing it this way actually account for errors made in the timesteps prior to the last? BPTT will backpropagate the error made at $X^i_{k_1}$ for $k_2$ timesteps, but what about the errors made at $X^i_{1...k_1-1}$ which were never calculated using softmax? Don't these matter just as much?

Thanks for your help!

[1] Mikolov, Tomáš. "Statistical language models based on neural networks." Presentation at Google, Mountain View, 2nd April (2012).

Answer 1

I've spent the past few days researching this and doing some experiments. It seems that evaluating the softmax only at the end of the sequence is less effective for predicting the next value in a sequence, but is necessary if your task involves classifying a sequence as a whole (sentiment analysis for example).

Here is a comparison of the error over time for only evaluating the softmax at the end of the sequence (shown in green) and evaluating it at every step in the sequence (shown in blue)

Here is an example I found of different types of architectures for different problems. The "many to many" would be evaluating an output for every step in the sequence, and the "many to one" would be only evaluating a final prediction. It all depends on your task.

I wrote a more in depth explanation as an answer to a similar question I asked here Why back propagate through time in a RNN?