In order to familiarize myself with semantic segmentation and convolutional neural networks I am going through this tutorial by MathWorks:

Semantic Segmentation Using Deep Learning

I did not use the pretrained version of Segnet since I wanted to test on my custom data set. All code is the same, however I have different classes, and fewer labels. Below image shows the label name and amount of pixels associated with each.

enter image description here

To make up for the low pixel data for class 2, median frequency balancing was performed.

imageFreq = tbl.PixelCount ./ tbl.ImagePixelCount
classWeights = median(imageFreq) ./ imageFreq

I proceed to train the network using the code provided in the example with the options and lgraph unchanged. The SegNet network is created with weights initialized from the VGG-16 network.

Unlike the example, I get a much lower global accuracy:

enter image description here

To gain further insight I plotted the Mini-batch accuracy and Mini-batch loss against each iteration.

enter image description here

It is clearly seen that the accuracy fluctuates wildly and ends up worse than it started, so the network learned absolutely nothing! However the loss decreased gradually.

A possible solution I propose would be to use inverse frequency balancing. However, in the example above, median frequency balancing was already performed, so I doubt how much this would help.

Is the terrible performance related to simply not having enough training data? Can anything be be done to improve performance with existing data? Any suggestions are greatly appreciated.

  • $\begingroup$ 100 iterations is not nearly enough to see any useful trends. Try training for 10000 $\endgroup$ – shimao Dec 1 '18 at 18:38
  • $\begingroup$ Valid point. I tested with 100 iterations since MathWorks usually uses 100-200 iterations in their examples. $\endgroup$ – Rrz0 Dec 1 '18 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.