Questions tagged [stochastic-policy]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
1 answer
243 views

Discrete and continuous actions in the same environment

I am working on a RL environment that requires both discrete and continuous actions as input from the agent. I currently have a fine implementation of DDPG which I would like to use for the continuous ...
0 votes
0 answers
35 views

What should be the policy for online reinforcement learning with intrinsic reward

An agent receives an extrinsic reward $r_{ext}$ and an intrinsic reward $r_{int}$ and a Q-function approximation is trained using TD learning such that $Q(s,a)$ approximates the expected return of $r_{...
0 votes
1 answer
108 views

Are the two $\epsilon$-greedy policies different?

I found 2 diffefent versions of $\epsilon$Greedy policy for monte carlo and q learning: For monte carlo: $\pi (a|s)=\epsilon /m +1-\epsilon$ to choose the best action and $\pi =\epsilon /m$ for other ...
2 votes
1 answer
375 views

Policy improvement in SARSA and Q learning

I have a rather trivial doubt in SARSA and Q learning. Looking at the pseudocode of the two algorithms in Sutton&Barto book, I see the policy improvement step is missing. How will I get the ...
1 vote
1 answer
78 views

How do measure how different two policies are?

I have two agents that both follow a baseline behavioral policy pi(a|s). If I then modify the state-action distribution for the two agents (resulting in two new policies), is there a standard measure ...
1 vote
0 answers
34 views

ε-Greedy with Q learning / SARSA can have stochastic policy?

Hello I'm now studying Q learning and SARSA with ε-Greedy , Softmax startegies. And have a question about my readings. In my readings, when SARSA with ε-Greedy, SARSA causes value-function ...
3 votes
1 answer
874 views

Proof that any $\epsilon-$greedy policy is an improvement over any $\epsilon-$soft policy

In the book by Richard Sutton and Andrew Barto, "Reinforcement Learning - An Introduction", 2ed edition, at page 101 there is a proof, and I don't understand 1 passage of it. We want to prove that ...
1 vote
0 answers
138 views

Why the Monte Carlo Control algorithm is written this way?

I am having trouble to understand this algorithm, since this is not how I would have written it. To me, we should first start to fix a policy. Then, we evaluate the Q values associated with this ...
8 votes
2 answers
9k views

Is a policy always deterministic in reinforcement learning?

In reinforcement learning, is a policy always deterministic, or is it a probability distribution over actions (from which we sample)? If the policy is deterministic, why is not the value function, ...