I am training a Conditional WaveGAN (1D DCGAN for audio) using WGAN-GP whose generator is of an auotencoder architecture. The network is trained to take an audio input, compress it, then decompress it into it's original waveform. I achieved reasonable accuracy after around 18 hours, but the reproduced audio quality was too low. So I left the network to keep training. However, after around 18 hours, the loss of both the critic and generator began violently oscillating at large magnitudes.
I have a couple of questions:
- What may have caused this unexpected behaviour? Did one of the adversaries learn something that the other couldn't identify, therefore causing a spiral of compounding bad gradient updates? Not enough training data (I had around 50minutes of audio broken down into 1 second chunks.)
- Prior to the spike in loss, do the critic and generator and critic loss graphs look "good"? My concern was that they both converged to 0 too quickly, before the generator was able to learn the finer, more subtle details of audio decompression. Is this a consequence of the critic being too slow to enfore such rules to the generator?
btw, I clipped the images, since some of the final datapoints reached a magnitude of ~1000. Also, the critic and generator had about 20 and 22 million parameters respectively.