# How can Deep Q Learning be applied to scenarios with rewards only received in a final step?

I am applying DQ Learning to a continuous action space with rewards received at the end of each trial. My agent is in a fixed 24step long setting where it receives the reward at the end of those 24 steps.

DQ Learning uses experience replay and randomly samples out of these areas. That would mean that if I placed all these actions into my replay buffer, the agent would learn 23/24 times that whatever action it takes, the reward is 0. And then it would associate whatever action it took in the last of the 24 steps as the rewarding action. However, that is not at all what I want to learn.

So, can it be that the DQN learning approach dissolved the idea of the agent being successfully considering "expected future reward" in its policy updates? Or did I misinterpret DQN learning?

I read the typical papers on this:

All of those play Atari games and I believe they continuously receive a reward, so they can just randomly sample an state-action-rew-state tuple because they get a reward for almost all actions (more or less).

• @pascalwhoop: I would advise against meddling with the reward scheme. Essentially by doing this you are inserting your own "expert" heuristics or prior knowledge for the agent. There may be better approaches for sparse reward schemes - e.g. prioritised sweeping for planning, or TD($\lambda$) to update more of each trajectory at once in Q learning. Although it depends what your purpose is? – Neil Slater Jun 9 '18 at 15:47