4
votes
Accepted
Can Q-learning or SARSA be used to find an stochastic policy?
Neither Q-learning nor SARSA define strictly how the policy should be derived from action-values, so you might be able to use them to learn some approximation to stochastic policies, if you used a ...
- 6,944
2
votes
Accepted
SARSA when policy is not epsilon greedy
As long as you can sample action choice from a fixed policy, then a variant of SARSA that used that policy (as opposed to anything based on the current Q values) would evaluate that policy. You can ...
- 6,944
2
votes
Accepted
Purpose of trace-decay parameter in eligibility traces
The discount factor $\gamma$ is applied to the scalar sum of future rewards. It changes how much a single measure of future reward will contribute to an update step.
The trace decay $\lambda$ is ...
- 6,944
2
votes
Accepted
Policy improvement in SARSA and Q learning
The current policy is derived in SARSA and Q learning from the current action values. It is always the $\epsilon$-greedy or greedy action choice according to $\text{argmax}_a Q(s,a)$. There is no need ...
- 6,944
1
vote
Deduce the Bellman equation from the Value and Q functions
Those euquations are simply wrong. The value-functions under an policy $\pi$ are defined as the expected amount of the total (discounted) return following policy $\pi$ from state $s$ onwards. Since ...
- 53
Only top scored, non community-wiki answers of a minimum length are eligible