2
$\begingroup$

I have seen the word "proper policy" in a bunch of reinforcement learning papers. Unfortunately, these papers did not provide any formal definition of what they mean by "proper policy". My guess is it might be something related to the learning rate.

I would appreciate if someone can help me better understand this notion.

Thanks.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Could you give a reference to a specific paper that uses the term? $\endgroup$
    – Neil Slater
    Commented Jul 11, 2018 at 14:41

1 Answer 1

Reset to default
1
$\begingroup$

A "proper policy" is

A policy that has a non-zero chance to finally reach the terminal state

In some episodic problems it is possible to construct policies that cannot complete an episode. Unless ruled out in a paper, these would be counter-examples to theories of convergence etc.

Usually any $\epsilon$-soft policy (i.e. a policy with $\frac{\epsilon}{|\mathcal{A}|}$ minimum probability of any action) should be a proper policy.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.