I'm trying to build a Multi-Label Semantic Segmentation model, but while training, when I'm looking at the validation set, I can see that one label is far far behind, and in the end, he is not getting good results.

I'm using a U-net model with resnet50 backbone, tried categorial-cross-entropy loss, dice-loss, jaccard loss, but couldn't find a way to leverage this class.

I tried to give this class more weight in the loss function, I even tried to give this class over 90% and still the states were low.

Someone have any Idea, method, a way of action, even tricks from his experience to help me with that issue? is it a common problem? or I need to check my basics?


1 Answer 1


When using multiple classes in segmentation, one must pay attention to the classes' representation in the data-set. The first thing to do would be to check the confusion matrix and see if the class that's falling behind gets confused with another class. If it is confused with a particular class then the issue is probably coming for the data set. However, if the class representation seem correct, you should check out the model itself.

I've worked with a similar U-net model for the same purpose and have used a manually weighted L2-loss and it seemed to do the trick. Maybe you can try that.

Another thing you can try out is to load encoder weights (for resnet50) pre-trained on ImageNet (They can be found online), this will ease your model recognition of the said class.

Hope this can help !


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.