I have a number of pretrained RL models (PPO2, ACER, ACKTR, ...) and I want to compare their behaviour in the environment. This includes their performance in respect of the episode-reward as well as different behavioural patterns they might have developed. I am looking for a meaningful metric to use for the comparison, so to say.
In particular, I am experimenting with the
BipedalWalker-v3 environment from OpenAI (which trains models on a walking-motion), but would like to also try this on other envs.
The data 'recorded' from the 'sensors' of one model interacting with the env looks like this:
The data is quite high dimensional and the patterns are somewhat repetitive. I assume different models will show different patterns, but want to use a somewhat analytical way to prove this and exactly measure how different they behave.
My main idea evolves around splitting the entire data into the individual walking cycles (which I tried and failed to do using FFT) and comparing the 'walk' of the different models from the single cycles.
Because this does not really work out, I want to ask if you can think of a better way to go about this task?