I want to have an agent run through a maze, but the agent should be blind. I.e., the agent does not know where he is, only the number of steps he has taken already.

My problem now is, when considering a Q-table, that the things the agent learns could be quite contradictory: Let's say a target could be reached from two sides and in one case the agent has to move to the right and in the other case, the agent has to move to the left. In both cases, if the agent moves in the wrong direction, he fails.

Now, in episode 1 the agent reaches the goal from the left, makes the step to the right and reaches the goal. In episode 2, the agent does a bunch of random steps and after the same number of steps reaches the target from the right side, but does (as learned in episode 1), a step to the right and fails.

This could be prevented by keeping a memory of the executed actions, but then the state grows crazy big. But without memory, the learning is really hard because all the experience gets very conflicting.

Has anyone experience with this?

  • $\begingroup$ Could you clarify what exactly you wish to let the agent observe. For instance, does the agent observe that it has run into a wall? What constitutes failure? You mention that the agent "fails", but there is no formal definition for success or failure in RL. For instance does "fails" equal episode ends (with maybe a negative reward)? What causes this - running into specific obstacles - running out of time? $\endgroup$ – Neil Slater Aug 29 '18 at 19:49
  • $\begingroup$ One other thing you could make concrete in your question: Is the aim to have the agent to learn optimal (presumably fastest) path through a single fixed maze, or to learn optimal policy for exploring mazes to find a goal in general (i.e. the latter option might have a different maze generated in each episode)? $\endgroup$ – Neil Slater Aug 29 '18 at 20:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.