1
$\begingroup$

I am trying to model simple 2d continuous distributions with GANs. Here, I focus on a 2d distribution following a ring structure. The architecture of my networks are:

def make_generator_model(input_dim):
    model = Sequential()
    model.add(Dense(32, activation='tanh', input_shape=(input_dim,)))
    model.add(Dense(32, activation='tanh'))
    model.add(Dense(32, activation='tanh'))
    model.add(Dense(32, activation='tanh'))
    model.add(Dense(2, activation='sigmoid'))  # output dim=2
    model.summary()
    return model
def make_discriminator_model():
    model = Sequential()
    model.add(Dense(32, activation='tanh', input_shape=(2,)))  # input data dim=2
    model.add(Dropout(0.3))
    model.add(Dense(32, activation='tanh'))
    model.add(Dropout(0.3))
    model.add(Dense(32, activation='tanh'))
    model.add(Dropout(0.3))
    model.add(Dense(32, activation='tanh'))
    model.add(Dropout(0.3))
    model.add(Dense(1, activation='sigmoid'))  # output dim=1 because discriminator
    model.summary()
    return model

I have trained my models to minimize the standard loss of GAN architectures, using Adam(learning_rate=5e-4) for 1000 epochs. I report the Discriminator average prediction per epoch on the GIF, separated between fake and genuine data. Blue points are the training data, orange points are simulated by the Generator.

Why can't my GAN even capture such a simple 2d distribution?

I have tried different architectures: more layers, less layers, more nodes, less nodes, larger learning rate, lower learning rates... and even the Wasserstein GAN architecture, in vain.

Any insight would be much welcome :)

image

$\endgroup$
3
  • $\begingroup$ IIUC, it doesn't converge. This is one of a couple of common problems with GANs. Maybe this helps: developers.google.com/machine-learning/gan/problems $\endgroup$
    – frank
    Jul 28, 2022 at 8:57
  • 1
    $\begingroup$ Thank you for the reference. I have tried the Wasserstein GAN in vain... This is very surprising that GANs can generate high-level realistic images, but have a hard time to approximate low dimensional simple distributions $\endgroup$ Jul 28, 2022 at 11:35
  • $\begingroup$ can you try much, much wider networks? Like width 2,000? $\endgroup$ May 9, 2023 at 17:09

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.