Derivation of expected value in REINFORCE policy gradient

Question

This is a derivation in the book "Reinforcement Learning, an Introduction, 2ed" for the REINFORCE algorithm.

By definition $q_\pi(s,a)=\mathbb{E}[G_t|S_t=s,A_t=a]$. I don't understand how the inner expectation vanish when going from line 2 to line 3. Why the following is true?

$$\mathbb{E}\left[\mathbb{E}[G_t|S,A]\frac{\nabla\pi(A_t|S_t)}{\pi(A_t|S_t)}\right]= \mathbb{E}\left[G_t\frac{\nabla\pi(A_t|S_t)}{\pi(A_t|S_t)}\right]$$

This question was made in another stack but in one step of the accepted answer it is assumed that the conditional expectation is wrt $S_t=s$ (a specific value of $S_t$) and not $S_t$ which is a random variable.

Derivation of Monte Carlo Policy Gradient for REINFORCE

Answer 1

I came across this very interesting website with lots of properties for conditional expected values: https://randomservices.org/random/expect/Conditional.html

One of the formulas is this:

$$r(X)\mathbb{E}[Y|X]=\mathbb{E}[r(X)Y|X]$$

That is, if you are conditioning on a random variable $X$, deterministic functions of $X$ may be brought inside the expected value operator. I suspect this may be done also when you condition on multiple random variables:

$$\pi(A,S)\mathbb{E}[G|S,A]=\mathbb{E}[\pi(A,S)G|S,A]$$

If this holds and $\nabla\log\pi(A|S)$ is a deterministic function of state and action then:

$$\mathbb{E}[\mathbb{E}[G|S,A]\nabla\log\pi(A,S)]=\mathbb{E}[\mathbb{E}[\nabla\log\pi(A,S)G|S,A]]$$

And by the law of iterated expectations this is:

$$\mathbb{E}[\nabla\log\pi(A,S)G]$$