I want to have an agent run through a maze, but the agent should be blind. I.e., the agent does not know where he is, only the number of steps he has taken already.
My problem now is, when considering a Q-table, that the things the agent learns could be quite contradictory: Let's say a target could be reached from two sides and in one case the agent has to move to the right and in the other case, the agent has to move to the left. In both cases, if the agent moves in the wrong direction, he fails.
Now, in episode 1 the agent reaches the goal from the left, makes the step to the right and reaches the goal. In episode 2, the agent does a bunch of random steps and after the same number of steps reaches the target from the right side, but does (as learned in episode 1), a step to the right and fails.
This could be prevented by keeping a memory of the executed actions, but then the state grows crazy big. But without memory, the learning is really hard because all the experience gets very conflicting.
Has anyone experience with this?