Timeline for What's done with the expectations in this proof?
Current License: CC BY-SA 4.0
3 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 7, 2018 at 16:05 | comment | added | Dennis Soemers | @olliejday The expectation of a product of lots of terms can be split up in all kinds of ways. Here they split it up into two particular expectations of products. I think what you are suggesting would be possible too, but the particular way in which they split it up simply happened to be convenient / necessary for subsequent steps in the proof. | |
| Sep 7, 2018 at 15:59 | comment | added | olliejday | I see, but if the policies are dependent on as you say the things in parentheses s_t and a_t, I don't see why the second expectation I've highlighted keeps both policies from t+1 to t+k-1?? Surely these could be taken out as well and the reward at time t+k as shown in the equation would only need policies at t+k-1? I'm not clear why later policies are taken out and earlier ones are kept in the expectation rather than just the policies at the timestep before the reward? | |
| Sep 7, 2018 at 15:24 | history | answered | Dennis Soemers | CC BY-SA 4.0 |