# Questions tagged [dynamic-programming]

The tag has no usage guidance.

14 questions
Filter by
Sorted by
Tagged with
23 views

### Add maximum time step to value iteration algorithm

What would a value iteration algorithm look like if I specify a maximum time step? For example, from a given state the environment does not reach a terminating state but instead should terminate ...
21 views

### Circular reference in states of the Bellman equation

I want to formulate a game of darts as a dynamic program. The goal is to minimize the number of darts thrown while reaching checkout. A dart player has a score s. If his score is s = ...
47 views

### Bellman equation / dynamic programming for darts

When you play darts, you can throw at 62 regions, z on the dartboard. Namely, the singles regions S1, ..., S20, the double regions D1, ..., D20, the treble regions T2, ..., T20 and the single- and ...
36 views

### Bellman Optimality Operator fixed point

I'm reading Szepesvári's book on RL. My question is concerning the proof of Theorem A.10 (p. 71). Theorem Let $V$ be the fixed point of $T^∗$ and assume that there is policy $π$ which is greedy w.r.t ...
28 views

### How does AlphaZero guarantee it could make consistent improvement?

I know the detail of AlphaZero. And in detail, I know it is improving by "policy iteration" mechanism. I found an answer that prove it can finally converge to optimal. But... Is it still ...
23 views

### Bellman equations and reinforcement learning

We need to define just a few things first: $(a_t, s_t) \in \mathcal{A} \times \mathcal{S}$ are the action and state at time $t$; $r(s_t, a_t)$ is the reward for taking action $a_t$ in state $s_t$; ...
597 views

### Choosing a changepoint detection algorithm

I've been reading up on changepoint algorithms (dynamic programming, Bayesian Online Changepoint detection, Hidden Markov Models, etc.) and am looking to implement an algorithm that has a certain set ...
66 views

### Efficiency of Viterbi Algorithm

Considering that we have a sequence of observed states $\{y_1, y_2, \dots, y_T \}$ of length $T$. We want to generate a path $\{x_1, x_2, \dots, x_T\}$, which is a sequence of states \$x_n \in S = \{...
221 views

### Value of the absorbing state in a MDP and greedy policy - Why choose to go to the absorbing state if state value is 0?

I was going through an example of a Markov Decision Problem and I got the optimal value function with the value iteration algorithm described in Sutton Barto. In this algorithm I chose to initialise ...
224 views

### calculate a quarterly average, a monthly data [closed]

I'm new to R, maybe this is very simple but I don't know how to do it I have the following data ...
4k views

### Dyna-Q Algorithm Reinforcement Learning

In step(f) of the Dyna-Q algorithm we plan by taking random samples from the experience/model for some steps. Wouldn't it be more efficient if we construct an MDP from experience by computing the ...
85 views

### Unique game problem (ML, DP, PP etc) [closed]

Looking for a solution to my below game problem. I believe it to be some sort of dynamic programming, machine learning, or probabilistic programming challenge, but am unsure... This is my original ...