2
$\begingroup$

As we unfold the network by time, the gradients of each time step is calculated. Are the recurrent gradients averaged or summed over the time steps in the Tensorflow implementation?

$\endgroup$

1 Answer 1

1
$\begingroup$

The gradients are averaged across time-steps. I would imagine this would be especially important if you had very large differences of input sequence length.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.