Structure of Generative Adversarial Networks (GAN) for mapping a simulation model

Question

There is a simulation model of a system that I want to map as a neural network to test if a better execution time can be achieved with similar accuracy.

The simulation model receives real-valued measurement data of its environment and generates a real-valued output from it.

The real-valued measurement data of the environment are limited, so I considered using a Generative Adversarial Network for training the Predictor.

The goal would be for the Generator to generate realistic inputs. These would then be fed into the simulation model to generate the output. Input and output would then be used to train the Predictor.

The overall goal is to get a Predictor with a high quality and a good generalization ability.

The goal of the Generator would be on the one hand to generate inputs as realistic as possible, but on the other hand also to generate inputs for which the Predictor is not yet so well-trained.

What would be the best way to do that? Would you possibly add a Discriminator in addition to the Predictor to force the Generator to generate realistic inputs?

Answer 1

So you want to learn a dynamics model of some system, and to improve the robustness of the model, you want to sample a wide range of inputs from a GAN.

While it's certainly possible to learn a dynamics model, I don't think it's a good idea to generate inputs exclusively with a GAN -- indeed, GANs are known for mode dropping and mode collapse, exactly what you don't want. I would advise using VAEs if you want to go down this path, since they're not as vulnerable to this problem.

There is a large body of work which generates synthetic data (for example, computer renderings of 3D scenes), and then uses conditional GANs to improve the photorealism of the renderings. This avoids mode dropping (since you're in full control of the computer rendering). Of course the downside is that it's up to the programmer to generate varied, interesting, and challenging synthetic data.

A complementary line of attack is data augmentation -- it's often possible to easily perturb a single input datapoint slightly in many interesting ways (for images: flipping, cropping, rotating, color adjustments, elastic deformation, adding rain/fog, etc) in order to synthesize new datapoints.