How to distinguish episodic task and continuous tasks? I am reading the chapter 3 of Sutton's book Reinforcement Learning: An Introduction and I am confused by episodic task and continuous tasks.
In reinforcement learning, how to distinguish episodic task and continuous tasks?
 A: A continuous task never ends. Which means you're not given the reward at the end, since there is no end, but every so often during the task.
For example, reading the internet to learn maths could be considered a continuous task.
An episodic task lasts a finite amount of time. For example, playing a single game of Go is an episodic task, which you win or lose. In an episodic task, there might be only a single reward, at the end of the task, and one option is to distribute the reward evenly across all actions taken in that episode.
In a continuous task, rewards might be assigned with discounting, so more recent reactions receive greater reward, and actions a long time in the past receive a vanishingly small reward. For example the reward could be geometric with distance in the past, with a discount factor $\lambda \in [0, 1]$.
A: A continuous task can go on forever, an episodic task has at least one finite state (i.e. an end of the game). Mathematically speaking an episodic task has a state with transition probability 1 to itself and 0 anywhere else.
This is highly interpretable, as gets clear in question 3.7 of the book:
Is a maze (with reward 1 for escaping, else reward 0) episodic? Technically yes, there is a final state but theoretically it can still go on forever (running in circles). Treating this as episodic to train an agent will lead to him never reaching the end in a fixed amount of rounds per game and fixed amount of games, therefore not training at all.
