The source you linked has a lot of confusing notation, so I'm not surprised you're confused. First, I'm going to break down the above equation into simpler ideas, based on what I've gathered from reading the surrounding equations and captions.
Section 4.3 talks about grouping together and separating out time steps in a given policy's execution time in order to make simplifying assumptions about how we can compute the rewards $R(\tau)$ for a given policy simulation trajectory, $\tau$.
Between equations (11) and (12), they explicitly discuss how the rewards accumulated over a given simulation trajectory $\tau$ (composed of $T$ individual time steps) can be atomically broken down into each time step's rewards being summed up:
$$
R(\tau) = \sum_{t=0}^{T-1}R(s_t,a_t)
$$
My understanding is that the following equation is a matter of simply substituting the "full trajectory" $\tau$ with time steps $[0,T]$ with a single timestep of the trajectory at $t'$. The notation for the reward of the single step at $t'$ is $r_{t'}=R(s_{t'},a_{t'})$. Thus, if we simply make the substitutions from equation (11), then we immediately arrive at
$$
\nabla_\theta \mathbb{E}_{\tau \sim \pi _ \theta}[R(\tau)]=\mathbb{E}_{\tau\sim\pi_\theta}\left[R(\tau)\sum_{t=0}^{T-1}\nabla_\theta \log{\pi_\theta(a_t|s_t)}\right]
$$
$$
\Downarrow \mathbf{Restriction ~ to ~~ } t' \Downarrow
$$
$$
\nabla_\theta \mathbb{E}_{\pi_\theta}[r_{t'}]=\mathbb{E}_{\pi_\theta}\left[r_{t'}\sum_{t=0}^{t'}\nabla_\theta \log{\pi_\theta(a_t | s_t)}\right]
$$
Now, the reason the author starts with the single time step is to break down any given simulation trajectory into past ($t\in[0,t')$), present ($t=t'$) and future ($t\in(t',T]$). After making this decomposition into rewards at individual timesteps, we can substitute the total decomposition of a complete trajectory $\tau$ into its component timesteps (which they do in eqn (12)). This comes from, once again, recognizing that we break down $R(\tau) = \sum_{t=0}^{T-1}r_t$:
$$
\nabla_\theta\mathbb{E}_{\tau\sim\pi_\theta}[R(\tau)]=\mathbb{E}_{\pi_\theta}\left[\sum_{t'=0}^{T-1}r_{t'}\right]=\mathbb{E}_{\pi_\theta}\left[\sum_{t'=0}^{T-1}r_{t'}\sum_{t=0}^{t'}\nabla_\theta \log{\pi_\theta (a_t|s_t)}\right]
$$
$$
\Downarrow \mathbf{Reindexing ~ the ~sum } \Downarrow
$$
$$
\nabla_\theta\mathbb{E}_{\tau\sim\pi_\theta}[R(\tau)]=\mathbb{E}_{\pi_\theta}\left[\sum_{t=0}^{T-1}\nabla_\theta \log{\pi_\theta (a_t|s_t)}\sum_{t'=t}^{T-1}r_{t'}\right]
$$
$$
\Downarrow \mathbf{Substituting ~}\sum_{t'=t}^{T-1}r_{t'}=G_t \Downarrow
$$
$$
\nabla_\theta\mathbb{E}_{\tau\sim\pi_\theta}[R(\tau)]=\mathbb{E}_{\pi_\theta}\left[\sum_{t=0}^{T-1}G_t\cdot\nabla_\theta \log{\pi_\theta (a_t|s_t)}\right]
$$
Which gives us exactly the gradient form that we desired in the previous section via Theorem 4.1. The most crucial part of this section is the re-indexing of the sum in the above equation block. It allows us to shift the effects of the gradient to only consider how changes to the policy would affect future time steps, rather than past time steps.
Let me know in the comments if you'd like me to expand on any sections of my response. Thanks!
