3
votes
Why are the value and policy iteration dynamic programming algorithms?
Have you seen Silver's lecture? Did you know Bellman coined the dynamic programming term, his first book was called "Dynamic programming" in 1957, see Wikipedia?
DP is an algorithm ...
3
votes
Accepted
Dyna-Q Algorithm Reinforcement Learning
Wouldn't it be more efficient if we construct an MDP from experience by computing the state transition probabilities and reward distribution from experience and solve it by dynamic programming?
...
- 6,944
3
votes
Accepted
Choosing a changepoint detection algorithm
The R package bcp seem to fulfill all of these (associated paper here). It returns the probability of change point at each index in your data, so you have to set a ...
- 2,538
2
votes
Efficiency of Viterbi Algorithm
In a Hidden Markov Model, the Viterbi algorithm is the right way to find the highest-probability sequence of hidden states $\bf x$, given your sequence of observations $\bf y$. It will find an exact ...
- 9,001
2
votes
Accepted
Why do not more data points reduce the error of Gaussian Process Regression?
Your data-generating process might have some inherited variability/noise. Ultimately that variability is irreducible. Putting this in the context of simple linear regression: where $y \sim N(X\beta, \...
- 46k
1
vote
Bellman Optimality Operator fixed point
Your reference link is broken, however, there's no theoretical reason you can immediately arrive at $T^∗V^∗=V^*$ merely by their definitions, where $T^*$ is assumed to be the Bellman optimality ...
- 1,245
1
vote
Choosing a changepoint detection algorithm
Another R package that meets these options (and can be implemented online) is changepoint.geo. It reduces the multivariate time series down to a bi-variate (angle ...
- 1,309
1
vote
Accepted
Value of the absorbing state in a MDP and greedy policy - Why choose to go to the absorbing state if state value is 0?
How is this fixed in general?
By having the reward function represent what you want the agent to achieve. If there is no differentiation in sum of rewards for any behaviour, then you have defined a ...
- 6,944
1
vote
Unique game problem (ML, DP, PP etc)
Note: I’m not sure if you would like a full answer (ie for someone to solve the entire problem for you) or hints for you to arrive at the solution. I will begin with a strong hint and if you want more ...
- 2,392
Only top scored, non community-wiki answers of a minimum length are eligible