3
$\begingroup$

My question is about implementing a DQN. I have seen some people use the actual rewards of the game and some clip it to [-1,1]. Is this just because they want to use their DQN for different Attari games and these games have different rewards or even if we only care about a specific game, it is good to divide all rewards to the maximum possible rewards to make them lie in [-1,-1]?

Thanks,

$\endgroup$

1 Answer 1

4
$\begingroup$

Clipping the rewards to lie in the [-1, 1] interval reduces the impact of extreme observations, making the model more robust. Another common approach is to clip the gradient of the loss for a minibatch so that no single step is too extreme. Whether or not these are helpful changes from case to case, but generally robust measures like these are helpful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.