# Discount factor for self play in reinforcement learning

Imagine we have a two player game with some sort of sparse reward (few actions are rewarded over the course of a game). My understanding is that ordinarily, the reward administered to actions would decrease by discount factor $\gamma$ each timestep as $t_\text{from reward}$ increases.

If we were attempting to learn a policy for a two player turn-based game (such as tic-tac-toe), a generally accepted strategy seems to be inverting the environment every other move such that the agent is always playing from the same perspective. However, if we naively administered reward for the agent's actions by our previous policy, we would end up rewarding self-destructive behavior of the agent every other timestep. Thus, it would make sense to me to give alternatingly negative reward.

Is it common practice to achieve this behavior by assigning a negative discount factor $\gamma$? Although this would make sense to me, I haven't seen examples where $\gamma < 0$ in published literature. Alternatively, is there a better way to set up the environment for these sorts of games such that we can still administer rewards as we would for single player games? Thank you.

• Interesting problem. Please, tell us what RL algorithm you consider. – Karel Macek Sep 22 '17 at 7:33