I have a dataset which contains the data from 10 classes. The classes look well-separated that the accuracy rate of different classifiers is more than 95%.
The goal is to change the data distribution in a way that they can’t be not easily classified. Some sort of noise should be added to the data that changes the distribution of the data but I am only allowed to change the input data with these actions: 1- increase the positive values 2- decrease the negative values 3- or no change to the feature
For example, if the input is [ +1, -4, +6, -8], one possible noise is [+2, -1, +1, -2], which the final input after adding this noise is: [+3, -5, +7, -10].
If we consider d as a random noise between [0,1], then the modified input is: X’ = X + sign(X) * d
The goal is to add the noise to the data that changes the data distribution to normal distribution that they are not well separated and can't be easily classified. BTW, the amount of noise added to the input should not be too much.
In order to learn such a noise, I am going to use some idea like GAN (Generative Adversarial Network). The generator learns the noise and the discriminator measures how the data is separable. It seems like a reverse GAN.
In GAN, we learn to change the noise distribution to the data distribution but here we change the data distribution to the normal distribution (I mean change the data in a way that they can't be classified).
The above figure shows the architecture. The discriminator tries to measure the separability of the data after adding noise, and generator learns how to add noises to the data to confuse the discriminator.
If X is the input with label y: y is categorical.
noise = generator(X) X_prime = X + tf.sign(X) * noise output = discriminator(X_prime) real = tf.reduce_sum(y * output, 1) # the probability of the true class label other = tf.reduce_max((1 - y) * output - y * 10000,1) # the maximum probability, among all other classes than true class
The discriminator tries to minimize this loss function
d_loss = tf.reduce_sum(tf.maximum(0.0, (other - real)))
And generator is minimizing two factors: 1- the L2 norm of X and X_prime 2- it tries to minimize the negative of the discriminator loss function.
l2dist = tf.reduce_sum(tf.square(X - X_prime)) G_loss = tf.reduce_sum(tf.maximum(0.0, (real - other)))) +C * l2dist
C is a tuning factor that does not allow one factor to outpower the other one.
I implemented this scenario but the generator does not learn the noise distribution and the discriminator can classify the noisy input with high accuracy. I even tried to train the generator much more often than discriminator not to allow the discriminator outpower the generator but it still does not work.
The main problem is that I first train the discriminator, then I train the generator. The generator learns how to add noise to the data to fool the previously trained discriminator but in the next round when I update (train) the discriminator with the noise data from generator they are separable. I mean the generator learns how to fool the discriminator but does not learn how to make the data inseparable.
The question is whether this model can learn the noise distribution added to the input to make the data distribution inseparable or not. Is my model is right or am I missing something in the model?