2
$\begingroup$

There exists crucial assumption in one (several) proof of the GAN paper that doesn't really make sense to me.

Let $X \sim p_{data}$ be the random variable associated with the input data, and $Z \sim p_{Z}$ be the random variable associated random noise. Note that they are written with bold letter in the paper e.g. $\mathbf{x}$.

In the computation of the optimal discriminator, in Proposition 1, the paper found that, https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf

$$D^{*}(X) = \frac{p_{data}(X)}{p_{data}(X) + p_{g}(X)}$$

where $p_{g}(X)$ is the probability distribution of the output of the random noise to the generator, $G(Z).$

However, to make this calculation work, the paper makes the key assumption that $X = G(Z) \sim p_g$.

That is, the random variable associated with the data $X$ is the same random variable as $G(Z)$.

However, in my opinion this is not true.

First, while $G(Z)$ represents a random variable takes on value in the same space as the data, they are is not exactly the same space. This means, $X: \Omega_1 \to \mathcal{X} \subseteq \mathbb{R}^n$, but $G(Z): \Omega_2 \to \mathcal{Y} \subseteq \mathbb{R}^n$, where $\mathcal{Y}$ could be a super or subset of $\mathcal{X}$. Furthermore, the sample spaces $\Omega_1, \Omega_2$ associated with the two random variables may differ as well. All this is to say that the random variables $X$, $G(Z)$ are not the same, hence we cannot make the argument that $G(Z) = X$, and proceed to calculate the optimal discriminator as shown in Proposition 1.

Also, at the notation level this is also troublesome, because $X \sim p_{data}$ is the random variable representing the data, but now $X \sim p_g$ as well.

All the above problems can be solved by denoting $G(Z)$ using a different random variable, say $X^\prime = G(Z)$. But the authors did not make this decision.

Therefore, I don't understand how equation (3) is derived.

Can anyone help me with this question?

$\endgroup$

1 Answer 1

1
$\begingroup$

I think the problem here is one of notation. The notation $X \sim p_G$ as used in the GAN paper is not meant to indicate $X$ is distributed according to $p_G$, but rather sampled from $p_G$. Of course this raises the problem about why the following equality holds in the proof:

$$ \mathbb{E}_{Z \sim p_Z}\left[ \log (1-D(G(z))\right] = \mathbb{E}_{X \sim p_G}\left[ \log (1-D(X)\right]$$

This is effectively a change of variables provided $G^{-1}$ exists. If not, one can use the result of the Radon–Nikodym theorem to change the measure.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.