Why does the GAN paper make the assumption that $G(Z) = X$?

Question

There exists crucial assumption in one (several) proof of the GAN paper that doesn't really make sense to me.

Let $X \sim p_{data}$ be the random variable associated with the input data, and $Z \sim p_{Z}$ be the random variable associated random noise. Note that they are written with bold letter in the paper e.g. $\mathbf{x}$.

In the computation of the optimal discriminator, in Proposition 1, the paper found that, https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf

$$D^{*}(X) = \frac{p_{data}(X)}{p_{data}(X) + p_{g}(X)}$$

where $p_{g}(X)$ is the probability distribution of the output of the random noise to the generator, $G(Z).$

However, to make this calculation work, the paper makes the key assumption that $X = G(Z) \sim p_g$.

That is, the random variable associated with the data $X$ is the same random variable as $G(Z)$.

However, in my opinion this is not true.

First, while $G(Z)$ represents a random variable takes on value in the same space as the data, they are is not exactly the same space. This means, $X: \Omega_1 \to \mathcal{X} \subseteq \mathbb{R}^n$, but $G(Z): \Omega_2 \to \mathcal{Y} \subseteq \mathbb{R}^n$, where $\mathcal{Y}$ could be a super or subset of $\mathcal{X}$. Furthermore, the sample spaces $\Omega_1, \Omega_2$ associated with the two random variables may differ as well. All this is to say that the random variables $X$, $G(Z)$ are not the same, hence we cannot make the argument that $G(Z) = X$, and proceed to calculate the optimal discriminator as shown in Proposition 1.

Also, at the notation level this is also troublesome, because $X \sim p_{data}$ is the random variable representing the data, but now $X \sim p_g$ as well.

All the above problems can be solved by denoting $G(Z)$ using a different random variable, say $X^\prime = G(Z)$. But the authors did not make this decision.

Therefore, I don't understand how equation (3) is derived.

Can anyone help me with this question?

Answer 1

I think the problem here is one of notation. The notation $X \sim p_G$ as used in the GAN paper is not meant to indicate $X$ is distributed according to $p_G$, but rather sampled from $p_G$. Of course this raises the problem about why the following equality holds in the proof:

$$ \mathbb{E}_{Z \sim p_Z}\left[ \log (1-D(G(z))\right] = \mathbb{E}_{X \sim p_G}\left[ \log (1-D(X)\right]$$

This is effectively a change of variables provided $G^{-1}$ exists. If not, one can use the result of the Radon–Nikodym theorem to change the measure.