In CNN, where the last layers are fully connected, how to make pixel-wise prediction to output an image(binary matrix), if the number of neurons in the last layer is less than the size of the image?

  • $\begingroup$ Welcome to Stats. Have a good time! $\endgroup$ May 22 '19 at 13:25

![enter image description here

The convolutional generation process is called unpooling and "deconvolution"(formally transposed convolutional layer) as shown above.

And we can also use RNN to generate pixels as stated in these paper: DRAW: A Recurrent Neural Network For Image Generation and Pixel Recurrent Neural Networks.

1. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
2. https://datascience.stackexchange.com/a/12110/9191

  • $\begingroup$ So there is no direct way to generate the segmented image, or this is the best option? $\endgroup$
    – Abdalrhman
    May 22 '19 at 12:46
  • $\begingroup$ @Abdalrhman You can also use RNN to generate please see my updated answer. I think there is no "direct" way to generate the image. $\endgroup$ May 22 '19 at 13:02

1 - U-net and similars

If your last few layers are fully connected, then they are, by definition, carrying global features of your images.

You can use transposed convolutional layers to obtain an image again, but you lose the fine detail in the image doing that.

To overcome that, the U-net and similar architectures were created. The idea is the same: convolution and downsampling (by using strides or pooling) followed by upsampling and convolutions (by using interpolation or tranposed convolutions), though there's an important detail. Layers in the downsampling network are connected, by means of skip connections, to layers with equal resolution in the upsampling network.

enter image description here

This will allow you to get an output map, as you desire, with preserved fine detail.

2 - Actually, tailored fully convolutional architectures will suffice

If you can do away with the fully connected layers (I suppose you're using these because you're doing transfer learning), you can use a fully convolutional architecture, where the only layers used are padded convolutions that preserve the dimensions of the input. You can also add skip connections, and also parallel branches with different receptive fields, adjusting padding accordingly.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.