# Questions tagged [value-iteration]

The tag has no usage guidance.

10 questions
Filter by
Sorted by
Tagged with
12 views

### How to modify the bellman operator for in-place iterative policy evaluation?

The iterative update rule for policy evaluation that is, approximating the value function for a given policy is: $$v^{k+1} = r_{\pi} + \gamma P_{\pi}v^{k}$$ This is the simultaneous update rule where ...
1 vote
17 views

### Gambling in multiple rounds with a maximum permitted bankroll and favorable or unfavourable probabilities

This is based on a deleted question, with the premises clarified to my understanding. You are gambling in a casino with particular rules: Bets are paid off at even amounts, so if you win a round you ...
• 40.4k
463 views

### Understanding Policy Iteration and Value Iteration

I have read an answer there but still cannot capture the ideas behind it. In Sutton's book, section Value Iteration, it is said that In fact, the policy evaluation step of policy iteration can be ...
• 23
283 views

### Why does First-Visit Monte Carlo Prediction (Policy Evaluation) converge?

In Barto and Sutton's "Introduction to Reinforcement Learning" book, in Section 5.1 (Monte Carlo Prediction), they describe the First-visit (and every-visit) Monte Carlo (MC) methods for ...
• 121
1 vote
87 views

### Q-value Iteration Convergence in Reinforcement Learning

I just started learning value iteration in reinforcement learning and I am confused about the theorem indicating that the iterations to have an error of at most $\epsilon$ grows with $λ$ is: N = \...
• 23
1 vote
183 views

### Add maximum time step to value iteration algorithm

What would a value iteration algorithm look like if I specify a maximum time step? For example, from a given state the environment does not reach a terminating state but instead should terminate ...
85 views

### Bellman equation / dynamic programming for darts

When you play darts, you can throw at 62 regions, z on the dartboard. Namely, the singles regions S1, ..., S20, the double regions D1, ..., D20, the treble regions T2, ..., T20 and the single- and ...
• 23
1 vote
43 views

### Small difference of q-function between different actions for the same state

I am trying out reinforcement learning using Q-learning. The data come from some made-up equations so I have infinite number of data. One thing that troubles me is after I learn the Q-function, I use ...
• 2,043
1 vote