2
$\begingroup$

I am working on a project training neural networks with an LSTM layer using Q-Learning.

I haven't been able to achieve optimal results on my test bench problems. I believe my problem has to do with the internal state of the LSTM.

When predicting an action using the network to modify the game state, the internal state is advanced. When using the network to make a predictions for updates to the network, the state is also advanced.

I am unclear on when exactly the internal LSTM states should be reset during the training process.

I was hoping someone might have a clear explanation or some links to some example code.

Thank you.

$\endgroup$
  • $\begingroup$ We need more information on your particular problem. The only thing I can definitely tell you: if your task is episodic, you should reset your LSTM state when the episode ends. $\endgroup$ – yobibyte Apr 19 '17 at 16:30
  • $\begingroup$ reset it when you want the network to discard the knoweledge about previous steps (e.g.: at the start of a new training epoch, after "game over" etc) $\endgroup$ – Bob Oct 31 '17 at 12:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.