I am reproducing the results from Hindsight Experience Replay by Andrychowicz et. al. In the original paper they present the results below, where the agent is trained for 200 epochs.

200 epochs * 800 episodes * 50 time steps = 8,000,000 total time steps.

enter image description here

I try to reproduce the resutls but instead of using 8 cpu cores, I am using 19 CPU cores.

I train the FetchPickAndPlace for 120 epochs, but with only 50 episodes per epoch. Therefore 120 * 50 * 50 = 300,000 iterations. I present the curve below:

enter image description here

and logger output for the first two epochs:

enter image description here

Now, as can be seen from my tensorboard plot, after 30 epochs we get a steady success rate very close to 1. 30 epochs * 50 episodes * 50 time steps = 75,000 iterations. Therefore it took the algorithm 75,000 time steps to learn this environment.

The original paper took approximately 50 * 800 * 50 = 2,000,000 time steps to achieve the same goal.

How is it that in my case the environment was solved nearly 30 times faster? Are there any flaws in my workings above?

NB: This was not a one off case. I tested again and got the same results.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.