How to use Hyperopt or perform hyperparamter tuning on a GAN? I am creating a Generative Adversarial Network, and I want to run Hyperopt on the GAN. However, I am confused about how to do this in Python in Keras because the GAN is essentially 2 models, with the output of one model, going into the next model. Any other method to run Bayesian Optimization or performing HP tuning on a GAN would be helpful.
 A: Unfortunately I don't think there is an easy way of doing this. It all comes down to this: you need some way to evaluate how good your GAN is, which isn't an easy task.
Why not use the discriminator?
The way evaluation is done during training is through the discriminator. So, one first thought would be to hold out one part of the training samples, generate the same amount of fake samples and check how many of those the discriminator predicts correctly. Generally speaking the goal is to achieve close to $50\%$ accuracy, i.e. the discriminator is cannot distinguish between real or fake.
This, however cannot work for evaluation. What you are evaluating is how confused the discriminator is. However, discriminator being bad doesn't imply that the generator is good (which is what you want).
Any viable solutions?
There are actually some theoretically-viable, but impractical solutions:

*

*Have a pretrained discriminator to evaluate all generators. To be able to compare generators, you need the same discriminator to evaluate all of them.

*Evaluate them through a secondary task. E.g. if you are generating cat and dog images, see how well an image classifier can classify cats as cats and dogs as dogs.

*Have some other domain-specific way of evaluating how realistic the fake samples are.

Unfortunately, none of these are straightforward to use in an automatic hyperparameter optimization scheme.
