0
$\begingroup$

I found 2 diffefent versions of $\epsilon$Greedy policy for monte carlo and q learning:

For monte carlo: $\pi (a|s)=\epsilon /m +1-\epsilon$ to choose the best action and $\pi =\epsilon /m$ for other actions

For q learning: $\pi (a|s)=1-\epsilon$ to choose the best action and $\epsilon$ to choose uniformly random action from possible actions

They both are stated as epsilon greedy policy. Are they different? (i think they are) am i missing somethings here or they really have the same name?

P/s: i am pretty sure they are different now, just aliitle confused about the names and the meanings of them in 2 different methods (monte carlo and qlearning)

$\endgroup$
3

1 Answer 1

0
$\begingroup$

$\epsilon$-greedy algorithm is taking the currently best policy with probability $1-\epsilon$ and other policy with probability $\epsilon$. The other algorithm you are describing is $\epsilon$-soft algorithm (the linked slides mention it under this name), a different algorithm, hence it uses a different rule.

$\endgroup$
5
  • $\begingroup$ Here specified the first one (in the slide) as epsilon greedy policy also medium.com/analytics-vidhya/… $\endgroup$
    – abcd
    Commented May 17, 2021 at 7:22
  • $\begingroup$ As i found here, the term "epsilon soft policy" only is about the least probability for choosing an action is $\epsilon$/|A(s)| stats.stackexchange.com/questions/342379/… $\endgroup$
    – abcd
    Commented May 17, 2021 at 7:23
  • $\begingroup$ @abcd the linked medium post mentions "epsilon greedy policies" and calls the policy "soft" (bolded in post). The $\epsilon$-greedy algorithm is just what I described, though as you learned from multiple sources, there are multiple modifications of this algorithm. The point of $\epsilon$-greedy algorithm is that there is a constant probability for choosing between exploration vs exploitation. $\endgroup$
    – Tim
    Commented May 17, 2021 at 7:31
  • $\begingroup$ Yeah it actually is a game of names:) and i agree that the second one "seems" better (it is widely used in q learning) but i dont know why The first one (i. E soft) still is used in monte carlo (as the medium link) $\endgroup$
    – abcd
    Commented May 17, 2021 at 7:34
  • $\begingroup$ @abcd each of those variants were designed to solve a particular problem. If you are considering a particular algorithm for your problem, you need to go through the literature & probably benchmark it against some simpler "default" solution first. $\endgroup$
    – Tim
    Commented May 17, 2021 at 7:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.