Can the GAN objective function be written as related to a log-likelihood of some “classical” statistical model?

Question

Can the objective function that GAN (Generative Adversarial Network) models optimize be written as a lower bound of the log-likelihood of some "classical" statistical model?

I am reading through the (as I understand) first description of a GAN, and a lot of the notation, vocabulary and writing/description conventions are unfamiliar to me. For posterity, the objective function is $$ \mathbb{E}_{x \sim p_{\text{data}}(x)}[\log D(x)] + \mathbb{E}_{z \sim p_{z}(z)}[\log (1 - D(G(z))]. $$ Notation:

• $x$: observed data vector
• $z$: standard normal variates
• $D$: a "discriminator" that maps data to the probability of that data being "real"
• $G$: the "generator" which maps standard normal variates into something that looks like data

Actually, to be totally honest, I'm not positive that this is the objective because, in addition to what I mentioned earlier, they appeal to game-theoretic concepts (which I'm not familiar with). Equation (1) of that same paper writes that $$ \min_G \max_D V(D,G) = \mathbb{E}_{x \sim p_{\text{data}}(x)}[\log D(x)] + \mathbb{E}_{z \sim p_{z}(z)}[\log (1 - D(G(z))]. $$ Is that right, though? Or should they be dropping the $\min$ and the $\max$ on the left hand side?

Answer 1

Do NOT take my word as gospel. That said, I think it's impossible. Or possible in a very specific way.

First problem is that D() and G() are not functions proper, they are variables. The second problem is that they are not independent. Worse, they are not only interdependent, they are recursively dependent on D-1(x) and G-1(x).

Note that the minmax on the left hand side you mentioned has subscripts, D and G. They are quite significant; what that equation is saying is that, from the system's POV, you're trying to find a pair of functions, where G(X) is the simplest possible function that causes the current D(X) to be wrong.

Let's say you pick D(x) = x1, then the optimal G+1 will be something like, say G(x1) := (1 - x1) := x2, which then forces D+1(x2) to be, say, D(x2) := x2+1 = x1+2... and that's just with univariate linear functions.

The only reason an idealized GAN stabilizes is that you've exhausted the limits of your architecture; otherwise, it's an infinitely iterated game of cops-and-robbers between two functions evolving to counter each other.