I have few doubts related to training Elman RNN using Backpropagation Through Time Algorithm.
Assume, I present a sequence to the network and the network adapts the parameters based on the error gradient including hidden state input(context units). Now, when I present the next sequence what should be the starting hidden state input(context units)? Should the hidden state input(context units) be the ones updated by the gradient decent one or the hidden state activation obtained by last input of the previous sequence?
PS: This is a follow-up question How to train Elman RNN for Temporal XOR?