I have the following architecture of discriminator in text to image synthesis where the image is convolved to lower dimension and concatenated with the text . My question is what is the use of concatenation ? 
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Could you please provide a link to the site where you found this diagram? It would be useful to understand the text of this diagram. $\endgroup$– Olivia WatkinsAug 4, 2019 at 17:53
-
$\begingroup$ In fact, this diagram is drawn in references to different sites like link, link and others. I understood all clearly. You can visit the links. Thank you @OliviaWatkins $\endgroup$– Sagar Budhathoki MagarAug 7, 2019 at 5:47
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
The goal of a discriminator is to assign a score predicting whether a given example is likely generated or real.
While it's difficult to be sure without seeing the tutorial/paper/code this diagram came from, my guess is that we concatenate the vector representation of the text with the representation of the image so the discriminator is able to make its prediction based on both the image and the related text.
If your discriminator were only provided the image, then your text-to-image model might learn to produce realistic-looking images which have nothing to do with the text they are given.
-
$\begingroup$ Thank you! @OliviaWatkins. Nice explanation. $\endgroup$ Aug 7, 2019 at 5:49
$\begingroup$
$\endgroup$
What you describe is a conditional GAN. Your condition (text input) provides some info to the generator of what to generate (i.e. image based on the given text).
The discriminator frequently uses the condition (text input) as well. This became quite popular, since this paper: https://arxiv.org/abs/1611.07004 . They used the concatenation successfully and this resulted in using the concatenation in the discriminator, especially in tasks related to images.
Since then, different ways of using this condition have been used, e.g. by inner product: https://arxiv.org/abs/1802.05637.
In short: Your schematic (probably) shows concatenating the text input in a later layer of the discriminator to facilitate its binary classification task (real/fake).
