Is it possible to train a NN to avoid the features that a different neural network finds? For example, let's train a simple 1 layer CNN with 1x1 kernels on a supervised binary classification problem. Then train a CNN with 2x2 kernel so that it doesn't learn any of the features from the 1x1 network?

Phrased another way, whatever the dominant feature of the 1x1 network is, I want my 2x2 network to avoid.

For example, say we were learning MNIST with just 0, 1 digits. However, all the zeros are in black, and the ones in white. So the 1x1 CNN just learns the color "if the picture is mostly black -> 1, else 0". It doesn't actually learn the shapes, just the average color. Now if I train a 2x2 CNN, it might learn both the color and something about the shape. Can I set this color dependance to zero? 'Unlearn' this feature?

  • $\begingroup$ You may want to look into generative adversarial networks where the descriminator network must learn new features which the generative network has not learned $\endgroup$ – Hugh Jun 29 '17 at 11:04

You could use an Inception Module. In essence you can try convolving with both kernels, 1x1 and 2x2 and concatenate together the outputs from each of them.
This way you are forcing a single neural network to learn different representations from each kernel and you don't need to train two different neural nets.
I recommend watching this brief video from Google's Deep Learning course about it:
https://classroom.udacity.com/courses/ud730/lessons/6377263405/concepts/63713420390923 . You can also find a more detailed explanation here:


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.