I am training a neural network for Audio classification.
I trained it on the UrbanSound8K dataset (Model1), and then I wanted to evaluate how different levels of added noise to the inputs influenced prediction accuracy. Baseline accuracy Model1 = 65%
As expected, higher levels of noise resulted in lower accuracy.
Then, I decided to perform data augmentation with noise (Model2). So I took the dataset, and I duplicated it with the same files but adding pink noise (+0 dB SNR) to them.
As expected (by me), the overall accuracy increased (a very tinny bit though, 0.5%), and the network became more robust to noise corruptions of the inputs.
However! One thing that I was not expecting was that now the network has reduced its accuracy when predicting only uncorrupted-with-noise inputs (validation inputs). Somehow, it has overfitted to the clean inputs, thus reducing prediction accuracy on these audios.
So, in numbers, Model2 predicts with 69% accuracy on noisy inputs (not necessarily the same noise that was trained with), and 47% accuracy on clean inputs.
Is there any explanation or intuition into this result?
I was expecting that the network, having now more and more varied training data, would learn more meaningful features. I guess it is more difficult to overfit to the noisy inputs, but still I don't understand why it has overfitted to the clean inputs mainly.
------------------------------------------------- EDIT 1 ----------------------------------------------------------------
Another piece of information that may be helpful:
Even when evaluating Model2 on noisy inputs with very little noise, still the network performs way better than on just clean inputs (which are very much the same as the inputs-with-little-noise to our ears)