Questions tagged [dynamic-programming]
The dynamic-programming tag has no usage guidance.
17
questions
0
votes
0
answers
20
views
Best betting strategy with positive EV while avoiding large loss
Coming from a financial stop loss background. Let's say in a game of $T$ rounds:
You start with $X_0=100$. And your profit $P=0$.
At round $t$, you can choose the bet size $z_t$. You will get $z_t\...
- 811
0
votes
0
answers
29
views
Predicting future states in hidden Markov models -- use the Viterbi algorithm?
The Viterbi algorithm is used to decode hidden states in hidden Markov models (HMMs) by working out which sequence of states is most likely. To do this, it first identifies which state $j \in \{1, ...,...
- 185
0
votes
0
answers
11
views
How to solve the stochastic dynamic programming problem by value function iteration?
I'm trying to learn value function iteration by myself. Recently I'm solving this following problem:
Consider a real business cycle model with production function: $y_t=e^{a_t}k_t^{\alpha}$, where $...
- 343
0
votes
0
answers
15
views
How to modify the bellman operator for in-place iterative policy evaluation?
The iterative update rule for policy evaluation that is, approximating the value function for a given policy is:
$$v^{k+1} = r_{\pi} + \gamma P_{\pi}v^{k}$$
This is the simultaneous update rule where ...
- 1
1
vote
0
answers
28
views
What Are Some Mathematical Functions that Can Represent Surge Pricing for Ride Hailing? [closed]
Consider a ride hailing service, where the fare needs to be dynamically adjusted. I know there are many machine learning driven models possible, but here I am looking for a simpler and sensible ...
- 543
1
vote
1
answer
127
views
Why do not more data points reduce the error of Gaussian Process Regression?
I am exploring using GP to approximate the value functions in some dynamic programming problems. Since all my data points are generated by myself, I can always train the GP with a larger dataset. ...
- 43
1
vote
1
answer
189
views
Add maximum time step to value iteration algorithm
What would a value iteration algorithm look like if I specify a maximum time step?
For example, from a given state the environment does not reach a terminating state but instead should terminate ...
0
votes
0
answers
85
views
Bellman equation / dynamic programming for darts
When you play darts, you can throw at 62 regions, z on the dartboard. Namely, the singles regions S1, ..., S20, the double regions D1, ..., D20, the treble regions T2, ..., T20 and the single- and ...
- 23
3
votes
1
answer
349
views
Bellman Optimality Operator fixed point
I'm reading Szepesvári's book on RL. My question is concerning the proof of Theorem A.10 (p. 71).
Theorem
Let $V$ be the fixed point of $T^∗$ and assume that there is policy $π$ which is greedy w.r.t ...
- 53
0
votes
0
answers
73
views
Bellman equations and reinforcement learning
We need to define just a few things first:
$(a_t, s_t) \in \mathcal{A} \times \mathcal{S}$ are the action and state at time $t$;
$r(s_t, a_t)$ is the reward for taking action $a_t$ in state $s_t$;
...
- 258
2
votes
2
answers
2k
views
Choosing a changepoint detection algorithm
I've been reading up on changepoint algorithms (dynamic programming, Bayesian Online Changepoint detection, Hidden Markov Models, etc.) and am looking to implement an algorithm that has a certain set ...
- 173
1
vote
1
answer
497
views
Efficiency of Viterbi Algorithm
Considering that we have a sequence of observed states $\{y_1, y_2, \dots, y_T \}$ of length $T$.
We want to generate a path $\{x_1, x_2, \dots, x_T\}$, which is a sequence of states $x_n \in S = \{...
3
votes
1
answer
968
views
Value of the absorbing state in a MDP and greedy policy - Why choose to go to the absorbing state if state value is 0?
I was going through an example of a Markov Decision Problem and I got the optimal value function with the value iteration algorithm described in Sutton Barto.
In this algorithm I chose to initialise ...
- 133
3
votes
1
answer
8k
views
Dyna-Q Algorithm Reinforcement Learning
In step(f) of the Dyna-Q algorithm we plan by taking random samples from the experience/model for some steps.
Wouldn't it be more efficient if we construct an MDP from experience by computing the ...
- 751
0
votes
1
answer
91
views
Unique game problem (ML, DP, PP etc) [closed]
Looking for a solution to my below game problem. I believe it to be some sort of dynamic programming, machine learning, or probabilistic programming challenge, but am unsure... This is my original ...
- 121
3
votes
0
answers
112
views
Understanding Approximate Dynamic Programming
I am trying to write a paper for my optimization class about Approximate Dynamic Programming. I found a few good papers but they all seem to dive straight into the material without talking about the ...
11
votes
1
answer
3k
views
Why are the value and policy iteration dynamic programming algorithms?
Algorithms like policy iteration and value iteration are often classified as dynamic programming methods that try to solve the Bellman optimality equations.
My current understanding of dynamic ...
