How to use BinaryCrossEntropy (intutively) for Generator Network in DCGAN model?

Question

TL;DR: Can someone tell me intuition working on BCE loss in the generator, specially for RGB as each pixel is having 3 values i.e. a list of values rather than just a single one.

I have same problem as this stack question but the thing is I am trying to find an intuition of BinaryCrossEntropy for RGB i.e 3 channels.

First, let me tell you that what I think of this function on Images (please correct if wrong), for example, we have a Grayscale image, for U-Net / Autoencoder architecture, you can Flatten the image and can say whether this Pixel is 0/1. That makes sense.

Also, when we are using U-Net Segmentation, we classify that this pixel belongs to one of which classes. So even if the last layer return WxHxC (say C = 6) channels, we can think it as that the pixel is One Hot Encoded.

And then just try to superimpose the class Number on the original image as:

The main thing here is that when we use the DCGAN Architecture, we use the BinaryCrossEntropy for Discirminator as well as the Generator network. Discriminator work is fine as it is very common but how does that work for Generator?

This Keras Implementation at the official site uses the same BCE loss function

Can someone tell me intuition working on BCE loss in the generator, specially for RGB as each pixel is having 3 values i.e. a list of values rather than just a single one.

Answer 1

Just to add some more mathematical details, the loss functions for DC-GAN are:

Generator loss: $$\ell_G = -\mathbb{E}_{z \sim p(z)}\left[\log D(G(z))\right]$$

Intuition: maximize the probability that the discriminator $D$ classifies image $G(z)$ as real (incorrect; minimized when $D(G(z)) \to 1$).

Discriminator loss: $$ \ell_D = -\mathbb{E}_{x \sim p_\text{data}}\left[\log D(x)\right] - \mathbb{E}_{z \sim p(z)}\left[\log \left(1-D(G(z))\right)\right]$$

Notation: We define that $x$ is a real image, $z$ is random noise. Furthermore, $p_{data}, p_z$ are the data-generating (i.e. real image) and noise distributions, respectively. Then $G, D$ are the generator and discriminator, respectively.

Intuition: maximize the probability of the discriminator $D$ correctly classifies real image $x$ (first term; minimized when $D(x) \to 1$) and the probability that discriminator $D$ correctly classifies fake image $G(z)$ (second term; minimized when $D(G(z)) \to 0$).

This is exactly consistent with that given by Goodfellow et. al. 2014, Algorithm 1 (p. 4), which is the objective DCGAN uses.

BCE Loss

These are all representible in terms of BCE losses. To see this, recall the definition of binary cross-entropy loss over some input distribution $\mathcal{P}$ and a model $f$ (assuming softmax/sigmoidal activation):

$$\ell_{BCE}(y, f(x)) = -y \log f(x) - (1-y) \log (1 - f(x)) $$

Let's break each term down. We'll assume we're working with one example at a time; this readily generalizes to the batched case.

1. The single term in $\ell_G$, or $-\mathbb{E}_{z \sim p(z)}[log D(G(z))]$, is $\ell_{BCE}(1, D(G(z))$.
2. The first term in $\ell_D$, or $-\mathbb{E}_{x \sim p_\text{data}}\left[\log D(x)\right]$ is similarly $\ell_{BCE}(1, D(x))$.
3. The second term in $\ell_D$, or $\mathbb{E}_{z \sim p(z)}\left[\log \left(1-D(G(z))\right)\right]$ is $\ell_{BCE}(1, 1-D(G(z)))$, which simplifies to $\ell_{BCE}(0, D(G(z))$.

Discussion

I think the main point of confusion is that the generator BCE loss isn't being applied directly to the image in this case, but rather between the logits of the discriminator with 1) the generated (fake) image as input vs. the real image as input.

You will also notice how generator gradients first have to propagate through the discriminator as well, such that (as explained previously) the generator updates are dependent on the discriminator's performance, which intuitively makes sense -- at a high level, depending on how poorly/well the discriminator is fooled, that should affect how we update the generator.

Do note that there are architectures that use a least-squares loss in the image space directly (i.e. LSGAN).

Explanations adapted from materials in Stanford CS231N: Convolutional Neural Networks for Visual Recognition. Highly recommended resource.

Answer 2

So as much as I have explored, and answered in this question, the loss is not for the generator but for the discriminator.

the flow goes in 2 steps like this:

1. Original Images (Concat) Generated Images -> Pass to Discriminator -> Calculate Loss based on BCE -> Calculate Gradients -> Update weights for Discriminator Network

2. Get Random Gaussian Noise -> Pass this Noise to Generator ->Generate Images -> Pass these Images to Discriminator -> Calculate BCE loss -> Calculate Gradients for Generator Network ONLY -> Update Weights ONLY for Generator.

So there no generator loss directly but the quality of images generated by generator is directly dependent on quality of images classified properly by the discriminator.

Total_Loss = D(Images) + D( G(Noise)) where D is Discriminator, G is Generator.

Please correct me if I am wrong.