I see the following equation in "In Reinforcement Learning. An Introduction", but don't quite follow the step I have highlighted in blue below. How exactly is this step derived?
Deriving Bellman's Equation in Reinforcement Learning
Amelio Vazquez-Reina
- 19.7k
- 27
- 81
- 120