Questions tagged [stochastic-policy]
The stochastic-policy tag has no usage guidance.
10
questions
0
votes
0
answers
32
views
What should be the policy for online reinforcement learning with intrinsic reward
An agent receives an extrinsic reward $r_{ext}$ and an intrinsic reward $r_{int}$ and a Q-function approximation is trained using TD learning such that $Q(s,a)$ approximates the expected return of $r_{...
- 1
0
votes
0
answers
29
views
How to prove that stochastic policy iteration converges?
I was reading Sutton's book Reinforcement Learning: An Introduction, especially policy iteration part.
There was a proof for convergence of policy iteration with deterministic policy.
So i tried to ...
- 1
0
votes
1
answer
98
views
Are the two $\epsilon$-greedy policies different?
I found 2 diffefent versions of $\epsilon$Greedy policy for monte carlo and q learning:
For monte carlo:
$\pi (a|s)=\epsilon /m +1-\epsilon$ to choose the best action and $\pi =\epsilon /m$ for other ...
- 1
2
votes
1
answer
343
views
Policy improvement in SARSA and Q learning
I have a rather trivial doubt in SARSA and Q learning. Looking at the pseudocode of the two algorithms in Sutton&Barto book, I see the policy improvement step is missing.
How will I get the ...
- 391
1
vote
1
answer
74
views
How do measure how different two policies are?
I have two agents that both follow a baseline behavioral policy pi(a|s). If I then modify the state-action distribution for the two agents (resulting in two new policies), is there a standard measure ...
- 111
1
vote
0
answers
31
views
ε-Greedy with Q learning / SARSA can have stochastic policy?
Hello I'm now studying Q learning and SARSA with ε-Greedy , Softmax startegies. And have a question about my readings.
In my readings,
when SARSA with ε-Greedy, SARSA causes value-function ...
- 11
1
vote
1
answer
219
views
Discrete and continuous actions in the same environment
I am working on a RL environment that requires both discrete and continuous actions as input from the agent. I currently have a fine implementation of DDPG which I would like to use for the continuous ...
- 11
1
vote
0
answers
131
views
Why the Monte Carlo Control algorithm is written this way?
I am having trouble to understand this algorithm, since this is not how I would have written it.
To me, we should first start to fix a policy. Then, we evaluate the Q values associated with this ...
3
votes
1
answer
843
views
Proof that any $\epsilon-$greedy policy is an improvement over any $\epsilon-$soft policy
In the book by Richard Sutton and Andrew Barto, "Reinforcement Learning - An Introduction", 2ed edition, at page 101 there is a proof, and I don't understand 1 passage of it.
We want to prove that ...
- 1,370
8
votes
2
answers
9k
views
Is a policy always deterministic in reinforcement learning?
In reinforcement learning, is a policy always deterministic, or is it a probability distribution over actions (from which we sample)? If the policy is deterministic, why is not the value function, ...
- 1,632