3
$\begingroup$


I am a beginner and it is my first question.
I know that Q-learning update equation is:
$Q(s_t, a_t) = Q(s_t, a_t)+α(r_{t+1} +γ·max_AQ(s_{t+1}, a_t)−Q(s_t, a_t))$
But in some of the researches it is changed as a slightly different version which will be called the Q-learning function from this point.
$Q(s_t, a_t) = r_{t+1} + γ · max_AQ′(s_{t+1}, a_{t+1})$

For example in a traffic control paper, which used deep Q-learning, it is used that different version.
I see also it, in other papers. Why it changed the Q-learning function?
Where is it useful to change?
Is it for the reason of never be negative?
Thank you

$\endgroup$

1 Answer 1

3
$\begingroup$

$\alpha$ is the learning rate parameter, and it is set to $1$ in the second application. It's perfectly possible that every application chooses/tunes its own learning rate. They're the same equation actually.

$\endgroup$
3
  • $\begingroup$ Alpha is one and Q(st,at) is deleted! why? $\endgroup$ Commented Feb 15, 2020 at 18:24
  • $\begingroup$ when you take alpha = 1, the one inside and the outside the parantheses cancel out; not deleted. $\endgroup$
    – gunes
    Commented Feb 15, 2020 at 18:27
  • $\begingroup$ Very good! OK! So it just focuses on new knowledge, and not the old Q-value! When is it useful to set the alpha as one? $\endgroup$ Commented Feb 15, 2020 at 20:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.