How can I deal with the Reinforcement learning problem with a fixed length episode but no terminal state? How can I deal with the Reinforcement learning problem with a fixed length episode but no terminal state?
Can I use the same method as the normal reinforcement learning with terminal state?
 A: You must define the states to match the learning objective you have set for the agent, and it is entirely possible that $\text{state} \neq \text{current_observation}$, when considering e.g. a map view on a grid world, or sensor readings etc as your $\text{current_observation}$. The state representation can include key metadata such as internal features of the agent, counters of events (such as numbers of times each action taken so far - or even a complete history to date), and current time. You should include any data in the state representation that affects future rewards. This is required to meet the assumption in reinforcement learning that the state has the Markov property.
Ending the episode is the terminal state regardless of representation. For most RL techniques that will mean:


*

*You actually stop the episode when running or simulating the environment

*The expected return is fixed at zero when calculating value functions from that point (this is important for boot-strapping methods such as TD learning)
In your specific case, as you have a fixed length episode:


*

*You should include the current time step (or time remaining) as part of the state, because that will heavily influence the return - i.e. the expected sum of all remaining rewards - from any given state.


This last point means effectively you do have a terminal state, it includes all state representations with the time set at the end.

This answer assumes that "a fixed length episode" is a deliberate part of the learning goals for the agent - your plan is that the agent must navigate the environment and collect maximum possible reward in a fixed time. The alternative is that you are limiting training length for some reason, but actually want to train an agent to maximise return in a continuous non-episodic environment - the answer does not address that.
