1
$\begingroup$

I have two agents that both follow a baseline behavioral policy pi(a|s). If I then modify the state-action distribution for the two agents (resulting in two new policies), is there a standard measure I can use to tell how "far" the policies are from the baseline policy, or each other?

More generally, Is there a standard way to measure the difference between two policies that operate on the same state space?

Divergence measures (like KL divergence) are useful, but are not actually distance metrics, so I'm wondering whether there's some known method I'm missing.

Improve this question
$\endgroup$
2
  • $\begingroup$ Hey posted the answer (no), and a link to that question. if you still think its without value I'll delete. $\endgroup$
    – Dirk
    Jan 28, 2021 at 18:54
  • 2
    $\begingroup$ Your question's on-topic here, so no need to delete: (1) it may attract further valid answers (the question on AI doesn't insist on "true" distances), & (2) it remains, for people searching on CV, as a pointer to the answer on the AI site (on which they may find much else of interest besides). $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Feb 2, 2021 at 16:40

1 Answer 1

Reset to default
0
$\begingroup$

It looks like the answer is, no, there not known distance metrics you can use to compare two policies.

Thanks to nbro for this question which is helpful and has a lot of good information about non-metric distances between policies: https://ai.stackexchange.com/q/25166/2444

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.