# does it make sense to define average reward in finite horizon

I am new to reinforcement learning but there is a situation I am considering using average reward instead of sum reward as objective for a finite horizon application problem.

Specifically, there are total of T possible time steps (e.g., usage rate of an app in each time-step), in each iteration, the reward may be 0 or 1. The goal is to maximize the daily average usage rate.

I see normally, the objective is defined maximizing $$\sum_{t = 0}^T \gamma^t r_t$$, I am wondering if I define the objective as one of the following two ways, is is normal (since I did not see any works using average reward in finite horizon)? And whether traditional Q-learning and DQN can be used un-modified?

alternative 1: $$1/T (\sum_{t = 0}^T \gamma^t r^t)$$, T varies is in each episode.

alternative 2: $$\frac{1}{T-t} \sum_{i = t - 1}^T \gamma^i r(s_i, a_i, s_{i+1})$$

• It doesn't make much sense to define average reward in a finite horizon setting since one can just redefine the reward functions $r_t$ to be the average (i.e., scale accordingly by $1/T$). Jun 25 at 16:51