13
$\begingroup$

I'm working on a convolution network for image recognition, and I was wondering if I could input images of different sizes (not hugely different though).

On this project: https://github.com/harvardnlp/im2markup

They say:

and group images of similar sizes to facilitate batching

So even after preprocessing, the images are still of different sizes, which makes sense since they won't cut out some part of the formula.

Are there any issues in using different sizes ? If there are, how should I approach this problem (since formulas won't all fit in the same image size) ?

Any input will be much appreciated

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

Are there any issues in using different sizes ? If there are, how should I approach this problem (since formulas won't all fit in the same image size) ?

It depends on the architecture of the neural network. Some architectures assume that all images have the same dimension, other (such as im2markup) don't make such an assumption. The fact that im2markup allow images of different widths don't bring any issue I believe, since they use an RNN that scans through the output of the convolution layer.

enter image description here

group images of similar sizes to facilitate batching

That's typically to speed things up by avoid adding too much padding.

Improve this answer
$\endgroup$
2
$\begingroup$

Have you considered simply scaling the images in the preprocessing stage? Intuitively, a human facing a scaled image will still be able to recognize the same features and objects, and there's no obvious reason why a CNN wouldnt be able to do the same thing on a scaled image.

I think that scaling the images to be the same size might be easier than trying to make a convolutional network handle images of different sizes, which I think would be up there in 'original research' land. You can certainly make the conv layers of a convnet handle images of any size, without retraining. However, the output of a convnet will typically be some kind of classifier, and this will probably work less well, if you feed in inputs of different size, I would imagine.

Another approach would be to just pad the images with zeros. But imagine intuitively you are looking at either a tiny photo, padded with black borders, or you can zoom in, so it subtends a reasonable arc in your visual field. Which would you do? Which is easier to see?

Improve this answer
$\endgroup$
4
  • $\begingroup$ doesn't scaling reduce the quality of the image introducing error and deforming features , if already the image is low resolution then scaling will decrease the quality of image to a point where even humans cannot recognize with ease but the unscaled image might be recognizable . $\endgroup$
    – Vikram Bhat
    Jul 12, 2017 at 8:20
  • 1
    $\begingroup$ do you have an example of an image that is recognizable to humans, unless you apply scaling, and then becomes unrecognizable? $\endgroup$
    – Hugh Perkins
    Jul 12, 2017 at 12:03
  • $\begingroup$ I'm wondering this too. When trying to use an existing trained model from keras, I'm thinking I could either scale the image up to fit InceptionV3 interface (299x299 from 32x32 -> CIFAR10). I think scaling loses quality. But I think the appropriate way of doing it would be to exclude the output FC and specify input shape to 32x32. I think this would require retraining though, as the input layer weights would be random. $\endgroup$
    – Joey Carson
    Mar 26, 2018 at 20:28
  • $\begingroup$ Scaling reduces quality but helps generalisation: there are many papers that noted a noticeable gain in recognition when Gauss blur was applied before training. Intuitively you have more different input samples that resemble a single "blurred" image, therefore making classification more robust. $\endgroup$
    – Matthieu
    Jun 16, 2019 at 21:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.