# Understanding step in proof of GAN algorithm convergence, involving convexity

I am reading the original paper on GANs, https://arxiv.org/abs/1406.2661. The proof of proposition 2, on the convergence of the gradient descent algorithm reads

Consider $$V(G, D) = U(p_g, D)$$ as a function of $$p_g$$ as done in the above criterion. Note that $$U(p_g, D)$$ is convex in $$p_g$$ ...

here (I think) $$V(G,D) = \mathbb{E}_{x \sim p_{\text{data}(x)}} [\log D(x)] + \mathbb{E}_{z \sim p_z(z)}[\log(1 − D(G(z)))]$$ is the value function of the GAN, and the 'above criterion' is that $$p_g$$ is updated so that the value function decreases (for the generator). Details are in the paper, it is not long.

What does it mean for $$U$$ to be convex in $$p_g$$? $$U$$ is a function of a distribution, and I can't interpret what convexity means in this context, even making assumptions on what $$p_g$$ is (e.g Gaussian)

What does it mean for $$U$$ to be convex in $$p_g$$? $$U$$ is a function of a distribution, and I can't interpret what convexity means in this context, even making assumptions on what $$p_g$$ is (e.g Gaussian)

If you recall the definition of convexity, it doesn't assume that function domain is finite-dimensional, only it is a convex subset of a linear space.

Distribution functions on $$A$$ are a subset of integrable functions on $$A$$, which is a linear space. It is in fact convex subset, just check the convexity property of $$\int_A p(x) dx = 1$$.

Because of that, it makes sense to say that some function of distributions on $$A$$ is convex.

• Thank you for clarifying the definition. I am happy that distribution functions are a convex subset. Am I right in thinking that $U$ is in fact linear in $p_g$, from which convexity follows (i.e, there's always equality in the convexity statement)? Mar 24, 2021 at 11:56
• Yes, expectation is linear operator w.r.t. density function. Mar 24, 2021 at 18:51

Note that $$V$$ is loss and it's actually Expectation.

And for Expectation we know $$E(aX + b) = aE(X) + b$$

Thus, by definition of convexity RHS ans LHS will be same.