For semantic segmentation problems, I understand that it's a pixel-wise classification problem. At the last layer of the neural network, I would basically have a 1x1x1 convolution layer with a softmax activation applied. The softmax activation essentially takes the depth-wise vector the output to generate probabilities summing to 1 (the highest probability represents the class at that pixel).

But what if I do multi-class segmentation in a single channel? Instead of the above, where I would have, say, foreground and background, what if I have classes 0,...n in a single channel? I would then label each pixel as either [0,....n], but how would the math work out in terms of the multi-class classification?

For instance, would I still be using softmax? I'm assuming not since softmax only sums to 1. I can't seem to find any references online, for some reason, for such a problem.

I realize that it's very popular to just have n channels for n classes, but it should also be possible to have a single channel with all the classes right?



1 Answer 1


I think that if you talk about just having two classes (like a binary segmentator) then one class output would suffice. I've seen a nice discussion about this in the pytorch forums here. Then you'd use a Sigmoid layer and the BCELoss.

The main reason might be accuracy, it's more difficult to have the right threshold-ed values for n classes. Say you have probabilities (that add up to 1) if you have say two classes then it's alright (example. if value <=0.5 then class 1, otherwise class 2), this of course becomes more complicated as the number of classes increases (for 10 classes, if value <=0.1 then class 1, if value >0.1 and <=0.2 then class 2, etc.). Increasing the amount of channels in the output and then doing a softmax it's just more accurate (I've tried one class outputs with subpar results).

Even for binary segmentators haven two channel outputs significantly outperformed a one channel output. Indeed it's a valid question and more insight would be required.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.