Timeline for policy iteration convergence
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 18, 2018 at 18:57 | comment | added | Ali Ghghgh | So it is really dependent on MDP because there is another proof in this book: "Dynamic programming and Markov processes" by Howard for completely ergodic systems in the finite horizon. And... the answer in the mentioned link wasn't covering my problem! | |
| Dec 18, 2018 at 13:36 | comment | added | Jesper for President | You could look at the article "CONVERGENCE PROPERTIES OF POLICY ITERATION" by MANUEL S. SANTOS and JOHN RUST, lemma 4.1 and 4.2 establishes monotone convergence. But offcourse they are considering a specific class of Markov Decision Problems. | |
| Dec 18, 2018 at 13:29 | comment | added | Jesper for President | I think your question is very unclear. After reading it I still do not know what you are aksing (and I actually spent 1 hour + trying to). I think you are asking for a proof of the monotonically improvement in value functions when doing policy iterations. But the way I see it the question you refer to explicitly adress this and you do not say why you think the answers are unsatisfying. If it is due to the lack of mathematical rigor I suggest you consult the litterature, there are articles written on the topic and getting monotone convergence is not trivial. | |
| Dec 18, 2018 at 7:34 | comment | added | Ali Ghghgh | If I need to explain more or anything else, please ask me. I really want to know the answer. Jumping from one value function to another and also jumping from one policy to another is making convergence. GPI figure shows it too but how? | |
| Dec 9, 2018 at 16:11 | comment | added | Ali Ghghgh | They are same but the question does not have only one part and I do not see any answer for last paragraph. For example, one of answers is covering getting stuck in local optimums. | |
| Dec 9, 2018 at 12:18 | comment | added | Jesper for President | How do the two questions differ? | |
| Dec 9, 2018 at 4:50 | review | First posts | |||
| Dec 9, 2018 at 7:00 | |||||
| Dec 9, 2018 at 4:46 | history | asked | Ali Ghghgh | CC BY-SA 4.0 |