0
$\begingroup$

I've tried several workarounds, such that my current approach uses a fully convolutional generator, Wasserstein loss, spectral normalization, TTUR, differential learning rates, and warm restarts. (Note that I don't consider myself a data scientist but rather a developer using the available tools to implement a feature.) Still, these same two failure modes keep arising, even with relatively small variations in parameters. Subjectively, I'd say that mode collapse is the more common problem -- e.g., output may seem relatively good at one stage, but continued training will lead to mode collapse. Conversely, when I find a combination of parameters that prevents mode collapse, it is often the case that training progresses to a point where it simply stops improving (and unfortunately this stopping-point doesn't produce clean enough images). Generally, when the convergence failure arises, the discriminator loss is significantly lower than the generator, however, the generator loss doesn't really increase, it just stays roughly the same. I'm curious whether the problem may be related to the fact that, ideally, my application requires a relatively large number of classes/clusters -- 400, or so. With a lower cluster count it does tend to perform better. I've also wondered whether perhaps some sort of data imbalance might be leading to these problems... (?)

Aside from hoping that someone here might have some advice, I'm curious whether there are any models that offer similar functionality to ClusterGAN? I have tried VaDE, for example, but it tends to fail during pretraining (fitting the GMM), due to the large requested number of clusters. What's a bit unusual about my application is that I need to infer a class (cluster) and latent from novel inputs, then generate images based on the inferred class, with the latent providing variation. So, unlike a lot of applications, I'm not discarding the encoder at runtime, but rather using it for inference.

I could imagine a two-step process, whereby I classify my training inputs in advance (though not ideal, this is possible with my data), then train a classifier for inference and a cGAN (or InfoGAN) for generation. This, however, will remove the benefit offered by inferring the latent at runtime. So the ClusterGAN really provides the end-to-end behaviour I'm after, and therefore seems like the best solution for my problem. But of course, if I can't get it trained, it's not a solution at all.

Any thoughts or advice appreciated.

$\endgroup$

1 Answer 1

0
$\begingroup$

In case it's of use to anybody in future, the two changes that finally helped the model both avoid mode collapse and converge were: 1) adding more filters to my Generator, and 2) adding dropout (0.2) to my Discriminator. I'd seen the general advice of strengthening the Generator in several articles/posts on GAN training tips, but I only found one post that specifically mentioned adding dropout, which turned out to be the most significant change.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.