In Goodfellow et al.'s Deep Learning, the authors derive the gradients with respect to the parameters of a recurrent neural network with discrete outputs and a hyperbolic tangent activation function. The network is diagrammed in the figure below:

enter image description here

As a first step to doing backpropagation, they calculate the derivative of the loss function $L$ with respect to the output $\mathbf{o}^{(t)}$. On page 374, they write:

The gradient $(\nabla_{\mathbf{o}^{(t)}}L)_i$ on the outputs at time step $t$, for all $i$, $t$, is as follows: $$(\nabla_{\mathbf{o}^{(t)}}L)_i = \frac{\partial L}{\partial o_i^{(t)}} = \frac{\partial L}{\partial L^{(t)}}\frac{\partial L^{(t)}}{\partial o_i^{(t)}} = \hat{y}_i^{(t)} - \mathbf{1}_{i, y^{(t)}}. \tag{10.18}$$

Here, $L$ is the total loss for a given sequence of $\mathbf{x}$ values paired with a sequence of $\mathbf{y}$ values, which is a sum of the losses over all the time steps:

$$\begin{align} &\phantom{=} L(\{\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(\tau)}\}, \{\mathbf{y}^{(1)}, \ldots, \mathbf{y}^{(\tau)}\}) \tag{10.12}\\ &=\sum_t L^{(t)}\tag{10.13}\\ &=-\sum_t \log p_{\mbox{model}}(y^{(t)} | \{\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(t)}\}) \tag{10.14} \end{align}$$

Moreover, $\hat{\mathbf{y}}^{(t)} = \mbox{softmax}(\mathbf{o}^{(t)})$, and according to the appendix on notation, $\mathbf{1}_\mbox{condition}$ is $1$ if the condition is true, $0$ otherwise.

I see that $\frac{\partial L}{\partial L^{(t)}} = 1$ because the derivative is zero for all the terms in the sum in equation (10.13) except the one that we're interested in, in which case it is $1$.


  1. How did they get $\frac{\partial L^{(t)}}{\partial o_i^{(t)}}$?
  2. How do you express $\log p_{\mbox{model}}(y^{(t)} | \{\mathbf{x}^{(1)}, \ldots, \mathbf{x}^{(t)}\})$ mathematically as a function of $\mathbf{o}^{(t)}$ so you can calculate its derivative?
  3. How do you interpret the notation $\mathbf{1}_{i, y^{(t)}}$ since $i, y^{(t)}$ is not really a condition?

1 Answer 1

  1. This is the Gradient of the loss where the inputs to the loss are the result of applying a softmax. I don't remember the exact details, but this is just from doing the calculus to figure out the gradients in the product that makes up $\dfrac{\partial{L}}{\partial{o_i}}$ at a given time step.

  2. Apply the softmax. The $i^{th}$ element of the hypothesis $\hat{y}$ is $\hat{y}_i = \dfrac{o_i}{\sum_{i'}o_{i'}}$, where $o_i$ is an element of the raw output vector at a particular time step. The sum of the elements in the vector $o$ isn't necessarily 1, so it's not a probability distribution; $\hat{y}$ clearly is.

  3. Given $\hat{y}$, the outputted vector of probabilities, subtract 1 from the element for which the truth $gt$ is 1 and 0 otherwise (this amounts to just subtracting the ground truth vector from the vector $\hat{y}$). Eg, if $gt^{T}$ is [0,0,1] (eg, this means that this example is from the third class) and $\hat{y}^{T}$ is [0,1,0] (we misclassified it as the second) we get [0,1,0]-[0,0,1]=[0,1,-1]. So if $gt^{T}$ = $\hat{y}^{T}$, all entries are 0 (no cost). If $gt^{T} \neq \hat{y}^{T}$ we get -1,1 in the spots where it gave either a false positive or false negative; these nonzero elements are then backpropagated to fix the weights that caused it to miss. This is the gradient of the loss wrt the input $\hat{y}$ to the loss.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.