I am working on a project training neural networks with an LSTM layer using Q-Learning.
I haven't been able to achieve optimal results on my test bench problems. I believe my problem has to do with the internal state of the LSTM.
When predicting an action using the network to modify the game state, the internal state is advanced. When using the network to make a predictions for updates to the network, the state is also advanced.
I am unclear on when exactly the internal LSTM states should be reset during the training process.
I was hoping someone might have a clear explanation or some links to some example code.