1
$\begingroup$

I have a dataset which contains the data from 10 classes. The classes look well-separated that the accuracy rate of different classifiers is more than 95%.

The goal is to change the data distribution in a way that they can’t be not easily classified. Some sort of noise should be added to the data that changes the distribution of the data but I am only  allowed to change the input data with these actions: 1- increase the positive values 2- decrease the negative values 3- or no change to the feature

For example, if the input is [ +1, -4, +6, -8], one possible noise is [+2, -1, +1, -2], which the final input after adding this noise is: [+3, -5, +7, -10].

If we consider d as a random noise between [0,1], then the modified input is: X’ = X + sign(X) * d

reverse gan

The goal is to add the noise to the data that changes the data distribution to normal distribution that they are not well separated and can't be easily classified. BTW, the amount of noise added to the input should not be too much.

In order to learn such a noise, I am going to use some idea like GAN (Generative Adversarial Network). The generator learns the noise and the discriminator measures how the data is separable. It seems like a reverse GAN.

In GAN, we learn to change the noise distribution to the data distribution but here we change the data distribution to the normal distribution (I mean change the data in a way that they can't be classified).

The above figure shows the architecture. The discriminator tries to measure the separability of the data after adding noise, and generator learns how to add noises to the data to confuse the discriminator.

If X is the input with label y: y is categorical.

noise = generator(X) 
X_prime = X + tf.sign(X) * noise
output = discriminator(X_prime)


real = tf.reduce_sum(y * output, 1)  # the probability of the true class label
other = tf.reduce_max((1 - y) * output - y * 10000,1)  # the maximum probability, among all other classes than true class 

The discriminator tries to minimize this loss function

d_loss = tf.reduce_sum(tf.maximum(0.0, (other - real)))

And generator is minimizing two factors: 1- the L2 norm of X and X_prime 2- it tries to minimize the negative of the discriminator loss function.

l2dist =  tf.reduce_sum(tf.square(X - X_prime))
G_loss =  tf.reduce_sum(tf.maximum(0.0, (real - other)))) +C *  l2dist

C is a tuning factor that does not allow one factor to outpower the other one.

I implemented this scenario but the generator does not learn the noise distribution and the discriminator can classify the noisy input with high accuracy. I  even tried to train the generator much more often than discriminator not to allow the discriminator outpower the generator but it still does not work.

The main problem is that I first train the discriminator, then I train the generator. The generator learns how to add noise to the data to fool the previously trained discriminator but in the next round when I update (train) the discriminator with the noise data from generator they are separable. I mean the generator learns how to fool the discriminator but does not learn how to make the data inseparable.

The question is whether this model can learn the noise distribution added to the input to make the data distribution inseparable or not. Is my model is right or am I missing something in the model?

$\endgroup$
  • $\begingroup$ I don't see a question mark here. Assuming you're asking "how can I make this model work?": Probably either (a) the problem is just too hard to turn into your target noise with these restrictions, or (b) you're running into optimization issues with your GAN, which are rampant in the literature. We don't really know enough from your question to know which one is the case, and it's probably too broad of a question for this format in any case. $\endgroup$ – Dougal Apr 18 '18 at 1:32
  • $\begingroup$ @Dougal thanks I edited the question, The question is whether this model can learn the noise distribution added to the input to make the data distribution inseparable or not. Is my model is right or am I missing something in the model? $\endgroup$ – MOH Apr 18 '18 at 6:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.