How to write a Kullback–Leibler divergence in mathematically correct notation? Let we have random variables $X$, $Y$, $Z$ and a function $q: Z \to \mathbb{R}$. How to write a Kullback–Leibler divergence between $q(Z)$ and $p(Z|X)$ mathematically correct?

*

*If we write $D_{\text{KL}}(q || p)$, this is ambigous. What $p$ do we mean: $p(Z)$, $p(Z|X)$, or $p(Z|Y)$?


*If we write $D_{\text{KL}}(q(Z) || p(Z|X))$, this seems incorrect. It is written as if Z takes on a specific value, and a result $q(Z) \in \mathbb{R}$ is substituted into the formula, but it's not. Kullback–Leibler divergence is meashured between distributions, not between probabilities.
Second variant seems better. But is mathematical correctness achievable?
 A: It seems that your question stems from Section 9.4 in Pattern Recognition and Machine Learning by C. Bishop, where the author used upper-case letters $X, Z$ to denote both random variables and arguments of likelihood functions.  For the latter, more standard notations should be lower-case letters. This ambiguity may cause confusions.
Some descriptions in your post can be made more accurate as well:  $q$ is not "a function from $Z$ to $\mathbb{R}$", instead, it is the density (here, "density" is a generic term which may stand for either the pmf when $Z$ is discrete or the pdf when $Z$ is continuous) of the latent random variable $Z$, which is a function from $\mathbb{R}$ to $\mathbb{R}$. To stress that $q$ is the density associated with $Z$, a subscript "$Z$" can be attached to $q$, i.e., using $q_Z$ to denote the density of $Z$. When we need to indicate the argument of $q_Z$ (e.g., the value of $q_Z$ evaluated at $z$), it can be made more specific as $q_Z(z)$ (I personally do not recommend writing $q(Z)$ or $q_Z(Z)$, which would be easily understood as a new random variable but not a distribution function). By the same token, the precise meaning of the notation "$p(Z|X)$" is the conditional density function of $Z$ given $X = x$ (where $x$ is the observed data values).  When the joint density $f$ of $(X, Z)$ exists, the conditional density function of $Z$ given $X = x$, denoted by $p_{Z|X}(z|X = x)$ for the greatest clarity, is given by
\begin{align}
p_{Z|X}(z|X = x) = \frac{f(x, z)}{p_X(x)},
\end{align}
where $p_X$ denotes the marginal density of $X$. It is easy to verify that $p_{Z|X}(z|X = x)$ is a valid density.  Oftentimes, without losing the clarity, $p_{Z|X}(z|X = x)$ can be abbreviated as $p_{Z|X}(z|x)$, or when the arguments are not needed, as $p_{Z|X}$ (with this notation, remember that this is a function of $z$, not $x$).
Now that both $q_Z$ and $p_{Z|X}$ are probability densities, the KL divergence between them can be unambiguously denoted as $\boxed{D_{\text{KL}}(q_Z||p_{Z|X})}$, whose precise expansion is:
\begin{align}
D_{\text{KL}}(q_Z||p_{Z|X}) = \int_\mathbb{R} q_Z(z)\log\left(\frac{q_Z(z)}{p_{Z|X}(z|x)}\right)dz. \tag{1}
\end{align}
Note that as $x$ changes, $(1)$ is a function of $x$.
