Different input size for training and prediction in CNN for image segmentation?

Question

I’m relatively unexperienced when it comes to deep learning and I’m trying to reimplement a CNN architecture for segmentation of medical images based on a paper. In the paper they state that they use input images that are of size 448x448. Further they state that they crop random sub-images which are 224x224 in order to have more data to train on.

Due to my lack of experience I’m not sure what the most likely interpretation of this is. Does this mean that they have trained the network with input size 224x224 and when using it on unseen data they are cut the input images into 4 pieces and feed it to the network or do they resize the input layers to 448x448 and reuse the weights from the network trained on 224x224? Or is there some other more likely interpretation that I’m unaware of?

Answer 1

After learning more about this topic and communicating with the authors of the paper it turns out that the solution in my case was that neither downsampling nor splitting the image in to four pieces, but simple using the other image size.

This is possible due to the fact that the weights in a fully convolutional neural network (FCNN), which is what I was trying to implement, are not dependent on the input image size. A FCNN is a CNN which does not contain any dense (fully connected) layers and are common in tasks such image segmentation.

Each convolutional layer contains a number of filters/kernels which are moved across the image performing convolutions producing a feature map that is fed to the next layer in the network. Pooling layers and upsampling layers for example work in a similar way. The filters are arrays of a fixed size where each element is a weight. So due to the way the convolutions are performed the weights are (more or less) independent of the size of the input and since there are not dense layers present (which are not independent of input size) this makes it possible to have different input sizes for training and testing for instance.

To achieve this in Keras for example you can set the input shape to (None, None,3) which will avoid any error due to input size.

Answer 2

I am also relatively unexperienced. I think crop just means using a subset of the image for training. It is a common way of data augmentation when you need more training data but only have a limited number of images. For a 448X448 image, you can randomly get a lot of different 224X224 cropped sub-images. They can be any position within the original image.

As for different input size for training and prediction, can you also use a cropped size for the prediction? I think the sizes have to match.