Questions tagged [sarsa]

The tag has no usage guidance.

14 questions
Filter by
Sorted by
Tagged with
1 vote
24 views

Convergence of the SARSA algorithm

I'm trying to figure out the convergence of the SARSA algorithm, but I need help. In the article "On the Convergence of Stochastic Iterative Dynamic Programming" by Jakkola, Jordana and ...
• 11
784 views

Policy improvement in SARSA and Q learning

I have a rather trivial doubt in SARSA and Q learning. Looking at the pseudocode of the two algorithms in Sutton&Barto book, I see the policy improvement step is missing. How will I get the ...
• 393
349 views

Deduce the Bellman equation from the Value and Q functions

I am trying to derive/deduce the bellman equation using Value and Q-functions. I came only so far with understanding it and tried it myself in Latex: Why is the $V^*$ suddenly in $Q^\pi$ function? ...
1 vote
57 views

ε-Greedy with Q learning / SARSA can have stochastic policy?

Hello I'm now studying Q learning and SARSA with ε-Greedy , Softmax startegies. And have a question about my readings. In my readings, when SARSA with ε-Greedy, SARSA causes value-function ...
• 11
443 views

SARSA when policy is not epsilon greedy

I would like to clarify a doubt that I have regarding SARSA. SARSA can be used for optimal control when the policy to take action $a$ is epsilon greedy. Suppose that the policy to take action $a$ is ...
• 1,116
523 views

In SARSA and Q-learning algorithms in RL, is policy updated during the iteration for Q-value learning?

In the video by Prof Brunskill "Stanford CS234 winter 2019 lecture 4" for model-free control (https://www.youtube.com/watch?v=j080VBVGkfQ), at 57:49/1:17:45, the pseudo code for SARSA ...
• 21
421 views

Purpose of trace-decay parameter in eligibility traces

In TD/SARSA-lambda, eligibility traces are decayed after each step by multiplying by the discount rate and the trace-decay parameter. I understand that: The discount rate is used to reduce the value ...
• 103
1 vote
333 views

Expected SARSA, SARSA and Q-learning

I would much appreciate if you could point me in the right direction regarding this question about targets for approximate ...
• 111
1 vote
30 views

How sensitive is reinforcement learning to the neural network structure

I am trying out Sarsa deep reinforcement learning on OpenAI gym CartPole-v0 problem. The state has 4 continuous features and the action is binary with either 0 or 1. The state-action vector is then ...
1k views

Episodic Semi-gradient Q-learning for Estimating approximation of optimal action-value function

at page 244 of Sutton and Barto book on Reinforcement Learning (book) is described the pseudocode for episodic semi-gradient Sarsa, while it is never given a pseudocode for the corresponding episodic ...
• 133
97 views

Different algorithms categorized in reinforcement learning

For some time I am going through reinforcement learning, and have found a lot of diverse information specially in area of Policies (algorithms). I figured out that policies can be classified in On ...
1 vote
674 views

Differences between Sarsa and Q-learning control procedural algorithms

I am referring to pages 130-131 of Sutton and Barto book on Reinforcement Learning available here: book I don't understand the slight difference that there is between the two procedural algorithms ...
• 133
1 vote
100 views

Convergence criterion for R-learning algorithm

I'm trying to find a policy for a simple game using R-learning algorithm. I have a field with values (agent can move in 4 directions) and the goal is to get from starting point to finish point with ...
• 275