1
$\begingroup$

I am reading the original paper on GANs, https://arxiv.org/abs/1406.2661. The proof of proposition 2, on the convergence of the gradient descent algorithm reads

Consider $V(G, D) = U(p_g, D)$ as a function of $p_g$ as done in the above criterion. Note that $U(p_g, D)$ is convex in $p_g$ ...

here (I think) $V(G,D) = \mathbb{E}_{x \sim p_{\text{data}(x)}} [\log D(x)] + \mathbb{E}_{z \sim p_z(z)}[\log(1 − D(G(z)))]$ is the value function of the GAN, and the 'above criterion' is that $p_g$ is updated so that the value function decreases (for the generator). Details are in the paper, it is not long.

What does it mean for $U$ to be convex in $p_g$? $U$ is a function of a distribution, and I can't interpret what convexity means in this context, even making assumptions on what $p_g$ is (e.g Gaussian)

$\endgroup$
0
$\begingroup$

What does it mean for $U$ to be convex in $p_g$? $U$ is a function of a distribution, and I can't interpret what convexity means in this context, even making assumptions on what $p_g$ is (e.g Gaussian)

If you recall the definition of convexity, it doesn't assume that function domain is finite-dimensional, only it is a convex subset of a linear space.

Distribution functions on $A$ are a subset of integrable functions on $A$, which is a linear space. It is in fact convex subset, just check the convexity property of $\int_A p(x) dx = 1$.

Because of that, it makes sense to say that some function of distributions on $A$ is convex.

$\endgroup$
2
  • $\begingroup$ Thank you for clarifying the definition. I am happy that distribution functions are a convex subset. Am I right in thinking that $U$ is in fact linear in $p_g$, from which convexity follows (i.e, there's always equality in the convexity statement)? $\endgroup$ – Arthur Conmy Mar 24 at 11:56
  • $\begingroup$ Yes, expectation is linear operator w.r.t. density function. $\endgroup$ – Jakub Bartczuk Mar 24 at 18:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.