# Negative samples on multiclass neural network training

I want to train a deep neural network to classify images. In every implementation I have seen, multiclass training uses only the positive examples for each class.

Is there any way to utilize negative samples for the N image classes, without resorting to training N binary - classification networks?

By negative samples, I mean that all the images annotated with label x are negative samples for the class y (in the case when the class x is not a subset of y, and vice versa). We can use these negatives in binary classification, so is there a way to be used in a multiclass NN?

• What did you mean when say "Negative samples"? Can you give some example? – itdxer Feb 17 '15 at 13:33
• Appended question with the response. – npit Feb 17 '15 at 14:52
• Well, if your negative example belongs to some class X, then softmax (usual multiclass loss) automatically understands that it's a negative examples for every other class. – Artem Sobolev Feb 17 '15 at 15:28

If each sample can only belong to one class, then the usual cross-entropy loss understands that a positive example for class $2$ is a negative example of classes $1, 3, ... k$. That is, for any class $k$, all samples in each other class are "negatives" with respect to $k$.