I am trying to classify every pixel on a large image (satellite image ~ 6000x4000 pixels) as belonging to one of the 4 classes:"Cloud", "Thin Cloud", "Clear", "Shadow." To that extent, I have taken inspiration from the paper "Brain Tumor Segmentation with Deep Neural Networks" where I design a neural network that looks something like this:

TwoPathCNN Image

The idea is that the network above will predict the class of the pixel by processing the 31x31 patch centered on that pixel.

Training and testing on various patches have been carried out however the main problem is full-image inference. Since the image is so large, it takes almost 2 hours to extract a patch at each pixel and then process it to determine the pixel class. In the paper, however, the authors "[feed] as input a full image and not individual patches. Therefore, convolutions at all layers can be extended to obtain all label probabilities $p(Yij|X)$ for the entire image."

So, my questions are:

  1. If I run all the convolution layers on my image, will I get a 4x5970x3970 dim tensor.
  2. How do I then make the leap to pixel-level classification for every pixel in my 6000x4000 image?
  • $\begingroup$ Is it essential that you process each image in one go and are you strongly tied to this architecture? I think there are easier ways to solve this problem. $\endgroup$ – MachineEpsilon Nov 2 '17 at 5:12
  • $\begingroup$ Hi @NaN, yes, I am strongly tied to this architecture. Having said that, what might these easier ways be? As for processing the image in one go, I am mainly concerned about speed when classifying around ~6000x4000 or ~24,000,000 pixels.Currently, my naive way extracts and processes 24,000,000 patches in ~2 hours. So,I am looking to speed up this process, and the authors of the paper suggest that the full image be processed in one go. I am open to other suggestions,however. $\endgroup$ – Guru Rahul Pradhan Nov 2 '17 at 11:48

I might suggest you should have a look at this paper Fully Convolutional Networks for Semantic Segmentation.

This paper suggests architecture which can take input of undefined dimensions and produce segmentation heat maps.

| cite | improve this answer | |

Given the 4x5970x3970 image (call it out), you can get a 5970x3970 image labels with 0,1,2,3 at each pixel by:

labels[i,j] = argmax_k out[k,i,j]

This is smaller than the original 6000x4000 image because it cannot predict the class on the images on the border (which cannot fit a 31x31 box around them without going out-of-bounds).

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.