Encoder Decoder Network - Computerphile : At the very beginning of this video, Michael Pound goes on to say:

So it (encoder decoder network) makes no assumptions about the size of the input the number of parameters, it just adapts itself depending on the size of the input. Which for images you can imagine makes quite a lot of sense they change size quite a lot, but in most other ways it acts exactly like a normal deep network

(emphasis mine)

Visual representation of a convolutional encoder decoder for image segmentation:

enter image description here

What I don't understand is the following:

  • If the input layer is a convolutional layer, doesn't this mean that the number of input neurons are fixed?
  • How can we feed in different image sizes to the same convolutional neural network and still get correct image segmentation?

2 Answers 2


In a fully connected neural network, the input can't change size because the linear transform in the first layer $Wx+b$ wouldn't work anymore -- the weight matrix $W$ wouldn't be of the correct shape.

However, note that you can apply a convolution to an image of any size without needing to change the parameters in the filter. So there is nothing restricting the size of the input image.

It makes sense that the network can generalize to inputs of different shape -- you are still applying the same convolutional filters to the same feature maps, so why shouldn't the result be the same as before?

  • $\begingroup$ This is an old thread, but won't the convolution of the image be larger for larger images? Then wouldn't it be a problem for linear layers after the convoluational layer? $\endgroup$ Dec 28, 2021 at 17:30

Encoder part can reduce size of the image by 100 and decoder part will increase it back by 100. It doesn't matter what will be the size of the image as long as decoder makes reverses operations compare to the encoder. Convolutional parameters are shared across the image, so it doesn't matter what's the height and width of the image.

Also, just because you can propagate images through the network it doesn't mean that you will get reasonable answer for any image. If you propagate one image through the network and get nearly perfect segmentation, but increasing area for the same image by large factor, let's say 1000, will destroy this effect only because objects in the up-scaled image are so large compare to the average representation of the same objects in the training that it won't be able to capture them with fixed set of the convolutional layers.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.