What is minibatch discrimination? Can someone explain what minibatch discrimination is, in simple terms?
Here is the link to the original paper.
(Minibatch discrimination: Sec 3.2, page 3)
 A: A common failure mode of Generative Adversarial Networks (GANs) is known as single mode collapse (the 'Helvetica scenario'), where the generator fools the discriminator by outputting generated data that looks exactly the same across one batch. The discriminator then learns that the single mode being produced by the generator corresponds to fake data, and forces the generator to look for a different mode. This process goes on ad eternum. The issue is that the generator will never be able to produce many different samples, which is what a data generator is supposed to do. Instead, it will only be capable of producing one single output.
To solve this problem, a concept called 'minibatch discrimination' was invented. The idea is for the discriminator to take into account all the information present in each batch of data, instead of looking to a single input. The single mode collapse is then quite easy to spot, since the discriminator will understand that whenever all the samples in a batch are very close to eachother, the data is fake. This will force the generator to produce many different good outputs in each data batch.
In practice, what the authors of the paper you mentioned do is to calculate the L1 norm between elements of matrixes related to the different inputs, effectively quantifying the concept 'close'. The smaller the L1 norm, the more similar the data inputs are. This new information is then concatenated with the original input and fed into the next layer of the discriminator, such that the discriminator can change its behaviour accordingly.
Finally, I leave a question to other people that may know more about this subject: what is the best way to choose the dimensions of the transformation tensor T (called B and C)? How should one initialize its elements?
