Gradient with respect to outputs for recurrent neural network

Question

In Goodfellow et al.'s Deep Learning, the authors derive the gradients with respect to the parameters of a recurrent neural network with discrete outputs and a hyperbolic tangent activation function. The network is diagrammed in the figure below:

As a first step to doing backpropagation, they calculate the derivative of the loss function $L$ with respect to the output $\mathbf{o}^{(t)}$. On page 374, they write:

The gradient $(\nabla_{\mathbf{o}^{(t)}}L)_i$ on the outputs at time step $t$, for all $i$, $t$, is as follows: $$(\nabla_{\mathbf{o}^{(t)}}L)_i = \frac{\partial L}{\partial o_i^{(t)}} = \frac{\partial L}{\partial L^{(t)}}\frac{\partial L^{(t)}}{\partial o_i^{(t)}} = \hat{y}_i^{(t)} - \mathbf{1}_{i, y^{(t)}}. \tag{10.18}$$

Here, $L$ is the total loss for a given sequence of $\mathbf{x}$ values paired with a sequence of $\mathbf{y}$ values, which is a sum of the losses over all the time steps:

$$\begin{align} &\phantom{=} L(\{\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(\tau)}\}, \{\mathbf{y}^{(1)}, \ldots, \mathbf{y}^{(\tau)}\}) \tag{10.12}\\ &=\sum_t L^{(t)}\tag{10.13}\\ &=-\sum_t \log p_{\mbox{model}}(y^{(t)} | \{\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(t)}\}) \tag{10.14} \end{align}$$

Moreover, $\hat{\mathbf{y}}^{(t)} = \mbox{softmax}(\mathbf{o}^{(t)})$, and according to the appendix on notation, $\mathbf{1}_\mbox{condition}$ is $1$ if the condition is true, $0$ otherwise.

I see that $\frac{\partial L}{\partial L^{(t)}} = 1$ because the derivative is zero for all the terms in the sum in equation (10.13) except the one that we're interested in, in which case it is $1$.

Questions:

1. How did they get $\frac{\partial L^{(t)}}{\partial o_i^{(t)}}$?
2. How do you express $\log p_{\mbox{model}}(y^{(t)} | \{\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(t)}\})$ mathematically as a function of $\mathbf{o}^{(t)}$ so you can calculate its derivative?
3. How do you interpret the notation $\mathbf{1}_{i, y^{(t)}}$ since $i, y^{(t)}$ is not really a condition?

Answer 1

1. This is the Gradient of the loss where the inputs to the loss are the result of applying a softmax. I don't remember the exact details, but this is just from doing the calculus to figure out the gradients in the product that makes up $\dfrac{\partial{L}}{\partial{o_i}}$ at a given time step.

2. Apply the softmax. The $i^{th}$ element of the hypothesis $\hat{y}$ is $\hat{y}_i = \dfrac{o_i}{\sum_{i'}o_{i'}}$, where $o_i$ is an element of the raw output vector at a particular time step. The sum of the elements in the vector $o$ isn't necessarily 1, so it's not a probability distribution; $\hat{y}$ clearly is.

3. Given $\hat{y}$, the outputted vector of probabilities, subtract 1 from the element for which the truth $gt$ is 1 and 0 otherwise (this amounts to just subtracting the ground truth vector from the vector $\hat{y}$). Eg, if $gt^{T}$ is [0,0,1] (eg, this means that this example is from the third class) and $\hat{y}^{T}$ is [0,1,0] (we misclassified it as the second) we get [0,1,0]-[0,0,1]=[0,1,-1]. So if $gt^{T}$ = $\hat{y}^{T}$, all entries are 0 (no cost). If $gt^{T} \neq \hat{y}^{T}$ we get -1,1 in the spots where it gave either a false positive or false negative; these nonzero elements are then backpropagated to fix the weights that caused it to miss. This is the gradient of the loss wrt the input $\hat{y}$ to the loss.