# Need help proving policy improvement theorem for epsilon greedy policies

I'm slightly confused about the proof epsilon greedy policy improvement. This is a part of the proof:

\begin{aligned} q_{\pi}\left(s, \pi^{\prime}(s)\right) &=\sum_{a} \pi^{\prime}(a \mid s) q_{\pi}(s, a) \\ &=\frac{\varepsilon}{|\mathcal{A}(s)|} \sum_{a} q_{\pi}(s, a)+(1-\varepsilon) \max _{a} q_{\pi}(s, a) \\ & \geq \frac{\varepsilon}{|\mathcal{A}(s)|} \sum_{a} q_{\pi}(s, a)+(1-\varepsilon) \sum_{a} \frac{\pi(a \mid s)-\frac{\varepsilon}{|\mathcal{A}(s)|}}{1-\varepsilon} q_{\pi}(s, a) \end{aligned}

I'm stuck on the very first line itself. Why is $$q_{\pi}\left(s, \pi^{\prime}(s)\right) =\sum_{a} \pi^{\prime}(a \mid s) q_{\pi}(s, a)$$? I've looked at other similar questions on this site, but they dont seem to address this line of the proof.

If we are at state $$s$$, we take action $$a$$ with probability $$\pi'(a|s)$$ according to policy $$\pi'$$ and we compute the expected value.
• Yes, but I still don't understand why the action value of taking one action ($\pi'(s)$) in state $s$ is equal to the expectation of all action values in $s$. Mar 5, 2021 at 4:02
• it is not one action right, it is action $a$ with probability $\pi'(a|s)$. Mar 5, 2021 at 5:27