I am new to reinforcement learning but there is a situation I am considering using average reward instead of sum reward as objective for a finite horizon application problem.

Specifically, there are total of T possible time steps (e.g., usage rate of an app in each time-step), in each iteration, the reward may be 0 or 1. The goal is to maximize the daily average usage rate.

I see normally, the objective is defined maximizing $\sum_{t = 0}^T \gamma^t r_t$, I am wondering if I define the objective as one of the following two ways, is is normal (since I did not see any works using average reward in finite horizon)? And whether traditional Q-learning and DQN can be used un-modified?

alternative 1: $1/T (\sum_{t = 0}^T \gamma^t r^t)$, T varies is in each episode.

alternative 2: $\frac{1}{T-t} \sum_{i = t - 1}^T \gamma^i r(s_i, a_i, s_{i+1})$

  • $\begingroup$ It doesn't make much sense to define average reward in a finite horizon setting since one can just redefine the reward functions $r_t$ to be the average (i.e., scale accordingly by $1/T$). $\endgroup$
    – Erik M
    Jun 25 at 16:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.