Confusion over the notation of REINFORCE algorithm in Reinforcement Learning

Question

From this paper, https://arxiv.org/pdf/1806.09460.pdf, page 9

I am confused by the line "sample $z_k \sim p(z;\theta_k)$"

It seems that $z_k$ is a random variable, as explained here Why are probability distributions denoted with a tilde?.

However, it was then plugged into the next line

$$\vartheta_{k+1} = \vartheta_k + \alpha_k R(z_k) \nabla_\vartheta\log(p(z_k;\vartheta_k))$$

which produces a vector $\vartheta_{k+1}$, which is a deterministic parameter. This can only occur if $z_k$ is a number (not a random variable).

Is there some abuse of notation going-on here or is this correct?

Is there a better way of writing this algorithm?

Answer 1

When they say that they sample $z_{k}$, it means that they get a realization of the random variable $Z_{k}$, it means that $z_{k} = Z_{k}(\omega) \in \mathbb{R}$ for some $\omega \in \Omega$, where $\Omega$ is the space on wich you random variable is defined.

Note that I have used $Z_{k}$ in capital letters to denote the random variable and $z_{k}$ in low letters to denote its realization.