Skip to main content

Questions tagged [value-iteration]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

How to modify the bellman operator for in-place iterative policy evaluation?

The iterative update rule for policy evaluation that is, approximating the value function for a given policy is: $$v^{k+1} = r_{\pi} + \gamma P_{\pi}v^{k}$$ This is the simultaneous update rule where ...
Atharva's user avatar
1 vote
1 answer
17 views

Gambling in multiple rounds with a maximum permitted bankroll and favorable or unfavourable probabilities

This is based on a deleted question, with the premises clarified to my understanding. You are gambling in a casino with particular rules: Bets are paid off at even amounts, so if you win a round you ...
Henry's user avatar
  • 40.4k
2 votes
1 answer
463 views

Understanding Policy Iteration and Value Iteration

I have read an answer there but still cannot capture the ideas behind it. In Sutton's book, section Value Iteration, it is said that In fact, the policy evaluation step of policy iteration can be ...
k2pctdn's user avatar
  • 23
0 votes
1 answer
283 views

Why does First-Visit Monte Carlo Prediction (Policy Evaluation) converge?

In Barto and Sutton's "Introduction to Reinforcement Learning" book, in Section 5.1 (Monte Carlo Prediction), they describe the First-visit (and every-visit) Monte Carlo (MC) methods for ...
gwtw14's user avatar
  • 121
1 vote
1 answer
87 views

Q-value Iteration Convergence in Reinforcement Learning

I just started learning value iteration in reinforcement learning and I am confused about the theorem indicating that the iterations to have an error of at most $\epsilon$ grows with $λ$ is: $$N = \...
WilliamW's user avatar
1 vote
1 answer
183 views

Add maximum time step to value iteration algorithm

What would a value iteration algorithm look like if I specify a maximum time step? For example, from a given state the environment does not reach a terminating state but instead should terminate ...
Stephane Hatgiskessell's user avatar
0 votes
0 answers
85 views

Bellman equation / dynamic programming for darts

When you play darts, you can throw at 62 regions, z on the dartboard. Namely, the singles regions S1, ..., S20, the double regions D1, ..., D20, the treble regions T2, ..., T20 and the single- and ...
HJA24's user avatar
  • 23
1 vote
1 answer
43 views

Small difference of q-function between different actions for the same state

I am trying out reinforcement learning using Q-learning. The data come from some made-up equations so I have infinite number of data. One thing that troubles me is after I learn the Q-function, I use ...
DiveIntoML's user avatar
  • 2,043
1 vote
1 answer
2k views

Q-learning shows worse results than value iteration

I'm trying to solve the same problem with different algorithms (Travel max possible distance with a car). While using value iteration and policy iteration I was able to get the best results possible ...
Most Wanted's user avatar
11 votes
1 answer
2k views

Why are the value and policy iteration dynamic programming algorithms?

Algorithms like policy iteration and value iteration are often classified as dynamic programming methods that try to solve the Bellman optimality equations. My current understanding of dynamic ...
Karthik Thiagarajan's user avatar