Questions tagged [stochastic-policy]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
32 views

What should be the policy for online reinforcement learning with intrinsic reward

An agent receives an extrinsic reward $r_{ext}$ and an intrinsic reward $r_{int}$ and a Q-function approximation is trained using TD learning such that $Q(s,a)$ approximates the expected return of $r_{...
user avatar
  • 1
0 votes
0 answers
29 views

How to prove that stochastic policy iteration converges?

I was reading Sutton's book Reinforcement Learning: An Introduction, especially policy iteration part. There was a proof for convergence of policy iteration with deterministic policy. So i tried to ...
user avatar
  • 1
0 votes
1 answer
98 views

Are the two $\epsilon$-greedy policies different?

I found 2 diffefent versions of $\epsilon$Greedy policy for monte carlo and q learning: For monte carlo: $\pi (a|s)=\epsilon /m +1-\epsilon$ to choose the best action and $\pi =\epsilon /m$ for other ...
user avatar
  • 1
2 votes
1 answer
343 views

Policy improvement in SARSA and Q learning

I have a rather trivial doubt in SARSA and Q learning. Looking at the pseudocode of the two algorithms in Sutton&Barto book, I see the policy improvement step is missing. How will I get the ...
user avatar
  • 391
1 vote
1 answer
74 views

How do measure how different two policies are?

I have two agents that both follow a baseline behavioral policy pi(a|s). If I then modify the state-action distribution for the two agents (resulting in two new policies), is there a standard measure ...
user avatar
  • 111
1 vote
0 answers
31 views

ε-Greedy with Q learning / SARSA can have stochastic policy?

Hello I'm now studying Q learning and SARSA with ε-Greedy , Softmax startegies. And have a question about my readings. In my readings, when SARSA with ε-Greedy, SARSA causes value-function ...
user avatar
  • 11
1 vote
1 answer
219 views

Discrete and continuous actions in the same environment

I am working on a RL environment that requires both discrete and continuous actions as input from the agent. I currently have a fine implementation of DDPG which I would like to use for the continuous ...
user avatar
  • 11
1 vote
0 answers
131 views

Why the Monte Carlo Control algorithm is written this way?

I am having trouble to understand this algorithm, since this is not how I would have written it. To me, we should first start to fix a policy. Then, we evaluate the Q values associated with this ...
user avatar
3 votes
1 answer
843 views

Proof that any $\epsilon-$greedy policy is an improvement over any $\epsilon-$soft policy

In the book by Richard Sutton and Andrew Barto, "Reinforcement Learning - An Introduction", 2ed edition, at page 101 there is a proof, and I don't understand 1 passage of it. We want to prove that ...
user avatar
8 votes
2 answers
9k views

Is a policy always deterministic in reinforcement learning?

In reinforcement learning, is a policy always deterministic, or is it a probability distribution over actions (from which we sample)? If the policy is deterministic, why is not the value function, ...
user avatar