I have a number of pretrained RL models (PPO2, ACER, ACKTR, ...) and I want to compare their behaviour in the environment. This includes their performance in respect of the episode-reward as well as different behavioural patterns they might have developed. I am looking for a meaningful metric to use for the comparison, so to say.

In particular, I am experimenting with the BipedalWalker-v3 environment from OpenAI (which trains models on a walking-motion), but would like to also try this on other envs.

enter image description here

The data 'recorded' from the 'sensors' of one model interacting with the env looks like this:

enter image description here

The data is quite high dimensional and the patterns are somewhat repetitive. I assume different models will show different patterns, but want to use a somewhat analytical way to prove this and exactly measure how different they behave.

My main idea evolves around splitting the entire data into the individual walking cycles (which I tried and failed to do using FFT) and comparing the 'walk' of the different models from the single cycles.

Because this does not really work out, I want to ask if you can think of a better way to go about this task?

  • $\begingroup$ Also I could not really find adequate tags for my posts, so if you can think of a key-word... $\endgroup$
    – Philipp
    May 28, 2020 at 18:24


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.