How to create training data for CNN using remote sensing imagery

Question

Before I start with the issue I would like to touch base with some background information.

I had been working with Random Forest for classification of Remote Sensing data, here the classification was based on the pixel value of the remotely sensed imagery.

So creating training CSV dataset was not difficult where my attributes in the CSV files were [ClassCode, B1, B2, B3, B4]

Where the ClassCode was the number assigned to the class and B1, B2, B3, B4 were the pxiel values extracted from remote sensing data

Now I am moving forward with implementing deep learning using CNN on remote sensing data.

I have ground-truth sample defining my classes for classification. I have been reading on some sample code available online where training samples are created not based on single-pixel value but the entire image.

In major samples for example, https://towardsdatascience.com/a-simple-cnn-multi-image-classifier-31c463324fa

The training data is created using an entire image, in the example above its types of animals and the result generated is just a label stating if the imagery passed is animal type "chicken" where I was expecting the result would be imagery classifying all the pixel where chicken is seen in imagery.

My objective is to generate classified imagery from satellite data:

I have training data from ground-truth as polygons on the imagery above stating the classes, for example, vegetation, road, buildings, etc. And the output should be as following:

Now running the trained model on imagery is really simple the only challenge that I am facing as of now is how do I create training data to train the CNN model using the satellite imagery and training polygons.

Any help would be really appreciated!

Answer 1

This looks like a good application for semantic segmentation. In fact, there have been contests in the past that address something very similar to your problem (e.g. http://www2.isprs.org/commissions/comm3/wg4/semantic-labeling.html).

The goal in semantic segmentation is to classify each pixel in an image with its class. It's a fairly popular deep learning application these days. Typically, the training data comprises many pairs of an optical image (like you have) with a mask which specifies the class for each pixel in the image. To create a dataset for semantic segmentation, I'd use the optical image as an input, and generate masks like the one in your second picture to be the target output. I'd probably employ several semantic segmentation models available online and see which ones work best.

If you're looking for tutorials, a Google search for "semantic segmentation tutorial" turns up quite a few results. I even managed to find a few that are applied to aerial imagery! I won't post the links here, but I'm sure you can find them.

Finally, Papers With Code lists quite a few research papers (with code!) that may be useful to you.

Answer 2

As @scherm mentioned, your problem falls under instance/semantic segmentation. In this case, your data is simply the satellite imagery you already have and the corresponding labels are the polygons which describe the instances (a.k.a pixels) which needs to be predicted for future satellite imagery that the model was not exposed to before.