# Confusion over the notation of REINFORCE algorithm in Reinforcement Learning

From this paper, https://arxiv.org/pdf/1806.09460.pdf, page 9

I am confused by the line "sample $$z_k \sim p(z;\theta_k)$$"

It seems that $$z_k$$ is a random variable, as explained here Why are probability distributions denoted with a tilde?.

However, it was then plugged into the next line

$$\vartheta_{k+1} = \vartheta_k + \alpha_k R(z_k) \nabla_\vartheta\log(p(z_k;\vartheta_k))$$

which produces a vector $$\vartheta_{k+1}$$, which is a deterministic parameter. This can only occur if $$z_k$$ is a number (not a random variable).

Is there some abuse of notation going-on here or is this correct?

Is there a better way of writing this algorithm?

• $\vartheta_k$ is a random variable. Jan 3, 2020 at 16:07

When they say that they sample $$z_{k}$$, it means that they get a realization of the random variable $$Z_{k}$$, it means that $$z_{k} = Z_{k}(\omega) \in \mathbb{R}$$ for some $$\omega \in \Omega$$, where $$\Omega$$ is the space on wich you random variable is defined.
Note that I have used $$Z_{k}$$ in capital letters to denote the random variable and $$z_{k}$$ in low letters to denote its realization.