8
$\begingroup$

Can we give images with variable size as input to convolutional neural network for object detection? If possible, How can we do that?


But if we try to crop the image, we will be loosing some portion of the image and if we try to resize, then, the clarity of the image will be lost. Does it mean that using inherent network property is the best if image clarity is the main point of consideration?

$\endgroup$
3
$\begingroup$

There are a number of ways to do it. Most of these have already been covered in a number of posts over StackOverflow, Quora and other content websites.

To summarize, most of the techniques listed can be grouped into two classes of solutions, namely,

  1. Transformations
  2. Inherent Network Property

In transformations, one can look up techniques such as

  • Resize, which is the simplest of all the techniques mentioned
  • Crop, which can be done as a sliding window or one-time crop with information loss

One can also look into networks that have inherent property to be immune to the size of the input by the virtue of layer behaviour which builds up the network. Examples of this can be found in terms of,

  • Fully convolutional networks (FCN), which have no limitations on the input size at all because once the kernel and step sizes are described, the convolution at each layer can generate appropriate dimension outputs according to the corresponding inputs.

  • Spatial Pyramid Pooling (SPP), FCNs do not have a fully connected dense layer and hence are agnostic to the image size, but say if one wanted to use dense layer without considering input transformations, then there is a interesting paper that explains the layer in a deep learning network.

References:

  1. https://www.quora.com/How-are-variably-shaped-and-sized-images-given-inputs-to-convoluted-neural-networks
  2. https://ai.stackexchange.com/questions/2008/how-can-neural-networks-deal-with-varying-input-sizes
  3. https://discuss.pytorch.org/t/how-to-create-convnet-for-variable-size-input-dimension-images/1906

P.S. I might have missed citing a few techniques. Not claiming this to be an exhaustive list.

$\endgroup$
1
$\begingroup$

The convolutional layers and pooling layers themselves are independent of the input dimensions. However, the output of the convolutional layers will have different spatial sizes for differently sized images, and this will cause an issue if we have a fully connected layer afterwards (since our fully connected layer requires a fixed size input). There are several solutions to this:

1. Global Pooling: Avoid fully connected layers at the end of the convolutional layers, and instead use pooling (such as Global Average Pooling) to reduce your feature maps from a shape of (N,H,W,C) (before global pool) to shape (N,1,1,C) (after global pool), where:

N = Number of minibatch samples
H = Spatial height of feature map
W = Spatial width of feature map
C = Number of feature maps (channels)

As can be seen, the output dimensionality (N*C) is now independent of the spatial size (H,W) of the feature maps. In case of classification, you can then proceed to use a fully connected layer on top to get the logits for your classes.

2. Variable sized pooling: Use variable sized pooling regions to get the same feature map size for different input sizes.

3. Crop/Resize/Pad input images: You can try to rescale/crop/pad your input images to all have the same shape.


In the context of transfer learning, you might want to use differently sized inputs than the original inputs that the model was trained with. Here are some options for doing so:

4. Create new fully connected layers: You can ditch the original fully connected layers completely and initialize a new fully connected layer with the dimensionality that you need, and train it from scratch.

5. Treat the fully connected layer as a convolution: Normally, we reshape the feature maps from (N,H,W,C) to (N,H*W*C) before feeding it to the fully connected layer. But you can also treat the fully connected layer as a convolution with a receptive field of (H,W). Then, you can just convolve this kernel with your feature maps regardless of their size (use zero padding if needed) [http://cs231n.github.io/transfer-learning/ ].

$\endgroup$
-2
$\begingroup$

Yes, simply select an appropriate backbone network which doesn't rely on the size of the input image to be some precise value -- most networks satisfy this criteria.

$\endgroup$
  • 2
    $\begingroup$ You are not wrong, but your answer is not very informative - what about expanding it to explain why most modern CNN can work with variable size images? Also, what are the limits to this variability (for example, don't try to mix different size images in the same mini-batch...)? Most people coming from either old-fashioned MLPs (input length is fixed) or old-fashioned CNNs (AlexNet & VGG-1X), with their pesky Flatten layers, don't understand how the modern CNNs can in principle take images of any size. $\endgroup$ – DeltaIV Jan 24 at 7:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.