3
$\begingroup$

I am currently working on a project that involves classifying each image as Good/Bad/Failed. We have a working convolutional neural network approach that works decent.

I also have trained a Fully convolutional neural network to do segmentation. THe labels for the segmentation are the actual objects in the image. So the FCN can tell me which pixels are what. Part of segmentation is identifying damaged areas which contribute to the good/bad/failed criteria.

My questions is what is best way to create a classifier that basically segments the image (tells me what and where everything is) but also put a single meta-class on top of that. Given that I have segmentation labels and full image classification images (not the same images though). Is there a way to train a single network to do both?

I do not know the relationship between damaged areas and Bad/Failed criteria besides that if an image contains damage it is not good. Beyond that I do not know. It may be as simple as damage > Threshold -> Failed but I doubt it is that easy.

I am using Caffe for everything

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Yes, I suggest you stack the output of segmentation network with the input image, doing any resizing necessary, and then another CNN on top which performs the final classification. This way, the classification makes use of the segmentation performed by the FCN to help it predict.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ It doesn't seem like you'd need a second CNN necessarily. For example, if the segmentation is done by a U-Net, there's an downsample and and upsample piece for segmentation, but one should be able to classify the entire image based on the results of in-between pooling layers (probably those just from the down-sampler actually). $\endgroup$
    – Alex R.
    Aug 17, 2019 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.