# Reinforcement learning, question from Sutton's new book

In Sutton's new RL book, in chapter 3, there is an exercise

Exercise 3.6 Suppose you treated pole-balancing as an episodic task but also used discounting, with all rewards zero except for −1 upon failure. What then would the return be at each time? How does this return differ from that in the discounted, continuing formulation of this task?

My answer is if it is episodic with discounting, the return at each time step is $-\gamma^{K_1} -\gamma^{K_2} - \cdots -\gamma^{K_n}$ where $K_i$ is the number of total time steps until it fails at episode $i$.

And for continuous tasks with discounting, the return at each time step is $-\gamma^K$ where $K$ is the number of time steps until it fails.

$$G_t = R_{t+1} + \gamma \cdot R_{t+2} + \gamma^2 \cdot R_{t+3} + ... + \gamma^{T-1} \cdot R_{T}$$
where $$T$$ is the terminal state and $$\gamma$$ is the discount rate.
Given that all rewards zero ($$R_{t+1}, R_{t+2}, R_{t+3}$$) except for −1 upon failure ($$R_T$$), you can extract the expected return at each time, which is different than what you propose.