1
$\begingroup$

I have a created a Convolution Neural Network in Keras for an image classification problem. The purpose of the CNN is to identify whether either of the two classes (e.g. dog and cat) is present in each input image sample. Images may contain one or both of the classes in a single image - both a dog and a cat.

As there are more than two classes I'm using the categorical cross-entropy loss function.

I am unsure how to represent the output matrix for samples that do not contain any of the two classes, or both. Some inputs contain neither a cat nor a dog, while some contain both.

Part 1: Cases where neither class is present in the image

In cases where neither class is represented in the input image, should the output matrix have two or three columns and be represented as:

  • [1, 0] = Only dog
  • [0, 1] = Only cat
  • [0, 0] = Neither dog nor cat

Or

  • [1, 0, 0] = Only dog
  • [0, 1, 0] = Only cat
  • [0, 0, 1] = Neither dog nor cat

Part 2: Cases where multiple classes are present in the image

In cases where both classes are represented in a single image, should an additional column be added to represent those cases (obviously adding/modifying a column depending on the answer to Part 1) so that the output appears as:

  • [1, 0] = Only dog
  • [0, 1] = Only cat
  • [1, 1] = Both dog and cat

Or

  • [1, 0, 0] = Only dog
  • [0, 1, 0] = Only cat
  • [0, 0, 1] = Both dog and cat

How should I represent the output matrix for the empty/null or multiple cases, and are there advantages or disadvantages of either alternative?

$\endgroup$
1
$\begingroup$

If you have three mutually exclusive categories, then [1,0,0], [0,1,0], and [0,0,1] with categorical cross entropy makes sense.

If membership in multiple classes is possible then adding 1 for each applicable class is right, but in that case categorical cross entropy seems wrong and it is more multiple separate 1/0 targets (i.e. binary cross entropy for each). Whether you need the third entry depends on whether it is just the same as 0 for the first two categories (then it is redundant) or whether you could have [1,0,1] or [0,1,1].

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.