For a research project, I am modeling parts of the human visual system using a VGG16 conv-net. A prominent feature of the visual system is that there are parts that only detect/classify certain categories of images (e.g. faces, text) and stay relatively silent for images not belonging to this category (not-face, not-text).
I'd like to train the model in such a way that it will classify, using a standard one-hot encoding scheme, different faces belonging to celebrities, but output all zeros (or all small values) for non-faces. Which training criterion should I use? Normally, I'd use cross-entropy for classifying faces, but how to incorporate wanting all zeros for training examples that are not faces?