For semantic segmentation problems, I understand that it's a pixel-wise classification problem. At the last layer of the neural network, I would basically have a 1x1x1 convolution layer with a softmax activation applied. The softmax activation essentially takes the depth-wise vector the output to generate probabilities summing to 1 (the highest probability represents the class at that pixel).
But what if I do multi-class segmentation in a single channel? Instead of the above, where I would have, say, foreground and background, what if I have classes 0,...n in a single channel? I would then label each pixel as either [0,....n], but how would the math work out in terms of the multi-class classification?
For instance, would I still be using softmax? I'm assuming not since softmax only sums to 1. I can't seem to find any references online, for some reason, for such a problem.
I realize that it's very popular to just have n channels for n classes, but it should also be possible to have a single channel with all the classes right?