I've already read almost every Questions-answers and material related to DQN, deep reinforcement learning, but I'm struggling to start working on simulation.
First of all, I'm trying to code using Matlab, although the most commonly used tool is Tensorflow. So far, I've understood the procedure of DQN about how experience buffer and stochastic gradient descent works for function approximator using neural networks.
What I'm confusing is that when parameters(θ) are updated, is only one step moved to the descent direction(▽loss) of performance function (in here, the loss = (target Q - output Q)^2) in an iteration? in other words, new θ= θ- γ*▽loss, this updates only once.
or in a single iteration, parameters(θ) are iteratively updated until satisfying certain termination criterion such as ||▽loss||≤η (for some small η>0), and then the next epoch is started as selecting an action given the next state according to ε-greedy rule.
Does my question make sense? or If I'm missing some important aspect here, please improve my questions.