0
$\begingroup$

In an MDP / in Reinforcement Learning, the optimal policy is often defined as something similar to: a policy that maximizes the value of all states at the same time.

But I am not sure whether this refers to the "actual" value of the states, or the agent's "estimated" value of the states.

For example, an agent could explore an environment and estimate the values of certain states, using the Bellman equation. But the actual values may be different to these estimates, and the estimates will only converge to the actual values with sufficient exploration and sufficient optimisation of the MDP.

In the case that the estimated values are not the same as the true values, and hence, the estimated optimal policy is not (necessarily) the same as the true optimal policy, do we still call the estimated optimal policy the optimal policy? If not, what do we call this policy? Is it the greedy policy?

$\endgroup$

1 Answer 1

2
$\begingroup$

The optimal policy refers to the actual value, not the estimated value -- in fact, some reinforcement learning agents might not even attempt to estimate the value of a given state.

do we still call the estimated optimal policy the optimal policy? If not, what do we call this policy? Is it the greedy policy?

Yes, greedy policy is the commonly used term for picking the action which maximizes the estimated value.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.