2
$\begingroup$

I am reading the original paper on GANs, https://arxiv.org/abs/1406.2661. The proof of proposition 2, on the convergence of the gradient descent algorithm reads

Consider $V(G, D) = U(p_g, D)$ as a function of $p_g$ as done in the above criterion. Note that $U(p_g, D)$ is convex in $p_g$ ...

here (I think) $V(G,D) = \mathbb{E}_{x \sim p_{\text{data}(x)}} [\log D(x)] + \mathbb{E}_{z \sim p_z(z)}[\log(1 − D(G(z)))]$ is the value function of the GAN, and the 'above criterion' is that $p_g$ is updated so that the value function decreases (for the generator). Details are in the paper, it is not long.

What does it mean for $U$ to be convex in $p_g$? $U$ is a function of a distribution, and I can't interpret what convexity means in this context, even making assumptions on what $p_g$ is (e.g Gaussian)

$\endgroup$

2 Answers 2

0
$\begingroup$

What does it mean for $U$ to be convex in $p_g$? $U$ is a function of a distribution, and I can't interpret what convexity means in this context, even making assumptions on what $p_g$ is (e.g Gaussian)

If you recall the definition of convexity, it doesn't assume that function domain is finite-dimensional, only it is a convex subset of a linear space.

Distribution functions on $A$ are a subset of integrable functions on $A$, which is a linear space. It is in fact convex subset, just check the convexity property of $\int_A p(x) dx = 1$.

Because of that, it makes sense to say that some function of distributions on $A$ is convex.

$\endgroup$
2
  • $\begingroup$ Thank you for clarifying the definition. I am happy that distribution functions are a convex subset. Am I right in thinking that $U$ is in fact linear in $p_g$, from which convexity follows (i.e, there's always equality in the convexity statement)? $\endgroup$ Mar 24, 2021 at 11:56
  • $\begingroup$ Yes, expectation is linear operator w.r.t. density function. $\endgroup$ Mar 24, 2021 at 18:51
0
$\begingroup$

Note that $V$ is loss and it's actually Expectation.

And for Expectation we know $E(aX + b) = aE(X) + b$

Thus, by definition of convexity RHS ans LHS will be same.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.