3
$\begingroup$

I have coded a Elman RNN using BackPropagation Through Time. In order to check my implementation, I have chosen Temporal XOR(a sequence of binary digits with the third being the xor of previous two and the rest random).

I have questions regarding how to train the model for this input. I am confused with the following strategies,

  • Should I give input $x_i$, and ask it to predict the next number in the sequence $x_{i+1}$ and do the error backpropagation
  • Should I give input $x_i$ ask it predict $x_{i+1}$, and $x_{i+1}$ ask it predict $x_{i+2}$ and then perform error backpropagation
  • Should I give a sequence of $t$ inputs i.e, $x_i,\ldots,x_t$ and ask it to predict $x_{i+1},\ldots,x_{t+1}$ and then perform error backpropagation

Could anyone please help me find the right training strategy? Also, if there are other sample problems like XOR for MLP, please suggest me.

$\endgroup$
1
$\begingroup$

Thanks everyone. I found the classic paper by Elman "Finding Structure in Time". In this paper he clearly explains the process of training the network on Temporal XOR problem.

$\endgroup$

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

  • $\begingroup$ Welcome to the site. At present this is more of a comment than an answer. You could expand it, perhaps by posting a full citation & a summary of the information in the paper? You can also convert it into a comment. $\endgroup$ – gung May 13 '16 at 17:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.