Questions tagged [sarsa]
The sarsa tag has no usage guidance.
14
questions
1
vote
0
answers
24
views
Convergence of the SARSA algorithm
I'm trying to figure out the convergence of the SARSA algorithm, but I need help. In the article "On the Convergence of Stochastic Iterative Dynamic Programming" by Jakkola, Jordana and ...
- 11
2
votes
1
answer
784
views
Policy improvement in SARSA and Q learning
I have a rather trivial doubt in SARSA and Q learning. Looking at the pseudocode of the two algorithms in Sutton&Barto book, I see the policy improvement step is missing.
How will I get the ...
- 393
0
votes
1
answer
349
views
Deduce the Bellman equation from the Value and Q functions
I am trying to derive/deduce the bellman equation using Value and Q-functions.
I came only so far with understanding it and tried it myself in Latex:
Why is the $V^*$ suddenly in $Q^\pi$ function? ...
1
vote
0
answers
57
views
ε-Greedy with Q learning / SARSA can have stochastic policy?
Hello I'm now studying Q learning and SARSA with ε-Greedy , Softmax startegies. And have a question about my readings.
In my readings,
when SARSA with ε-Greedy, SARSA causes value-function ...
- 11
3
votes
1
answer
443
views
SARSA when policy is not epsilon greedy
I would like to clarify a doubt that I have regarding SARSA. SARSA can be used for optimal control when the policy to take action $a$ is epsilon greedy. Suppose that the policy to take action $a$ is ...
- 1,116
2
votes
1
answer
523
views
In SARSA and Q-learning algorithms in RL, is policy updated during the iteration for Q-value learning?
In the video by Prof Brunskill "Stanford CS234 winter 2019 lecture 4" for model-free control (https://www.youtube.com/watch?v=j080VBVGkfQ), at 57:49/1:17:45, the pseudo code for SARSA ...
- 21
0
votes
1
answer
421
views
Purpose of trace-decay parameter in eligibility traces
In TD/SARSA-lambda, eligibility traces are decayed after each step by multiplying by the discount rate and the trace-decay parameter.
I understand that:
The discount rate is used to reduce the value ...
- 103
1
vote
0
answers
333
views
Expected SARSA, SARSA and Q-learning
I would much appreciate if you could point me in the right direction regarding this question about targets for approximate ...
- 111
1
vote
0
answers
30
views
How sensitive is reinforcement learning to the neural network structure
I am trying out Sarsa deep reinforcement learning on OpenAI gym CartPole-v0 problem. The state has 4 continuous features and the action is binary with either 0 or 1. The state-action vector is then ...
- 111
0
votes
1
answer
1k
views
Episodic Semi-gradient Q-learning for Estimating approximation of optimal action-value function
at page 244 of Sutton and Barto book on Reinforcement Learning (book) is described the pseudocode for episodic semi-gradient Sarsa, while it is never given a pseudocode for the corresponding episodic ...
- 133
0
votes
2
answers
97
views
Different algorithms categorized in reinforcement learning
For some time I am going through reinforcement learning, and have found a lot of diverse information specially in area of Policies (algorithms).
I figured out that policies can be classified in On ...
- 133
1
vote
1
answer
674
views
Differences between Sarsa and Q-learning control procedural algorithms
I am referring to pages 130-131 of Sutton and Barto book on Reinforcement Learning available here: book
I don't understand the slight difference that there is between the two procedural algorithms ...
- 133
1
vote
0
answers
100
views
Convergence criterion for R-learning algorithm
I'm trying to find a policy for a simple game using R-learning algorithm. I have a field with values (agent can move in 4 directions) and the goal is to get from starting point to finish point with ...
- 275
2
votes
1
answer
826
views
Can Q-learning or SARSA be used to find an stochastic policy?
If the optimal policy is known to be stochastic (e.g. like in the stone, paper, scissors game), can this stochastic policy be found using SARSA or Q-learning, or is it only possible with policy ...
- 33