Can a convolutional neural network take as input images of different sizes? I'm working on a convolution network for image recognition, and I was wondering if I could input images of different sizes (not hugely different though).
On this project: https://github.com/harvardnlp/im2markup
They say:
and group images of similar sizes to facilitate batching

So even after preprocessing, the images are still of different sizes, which makes sense since they won't cut out some part of the formula.
Are there any issues in using different sizes ? 
If there are, how should I approach this problem (since formulas won't all fit in the same image size) ?
Any input will be much appreciated
 A: 
Are there any issues in using different sizes ? If there are, how should I approach this problem (since formulas won't all fit in the same image size) ?

It depends on the architecture of the neural network. Some architectures assume that all images have the same dimension, other (such as im2markup) don't make such an assumption. The fact that im2markup allow images of different widths don't bring any issue I believe, since they use an RNN that scans through the output of the convolution layer.


group images of similar sizes to facilitate batching

That's typically to speed things up by avoid adding too much padding.
A: Have you considered simply scaling the images in the preprocessing stage? Intuitively, a human facing a scaled image will still be able to recognize the same features and objects, and there's no obvious reason why a CNN wouldnt be able to do the same thing on a scaled image.
I think that scaling the images to be the same size might be easier than trying to make a convolutional network handle images of different sizes, which I think would be up there in 'original research' land.  You can certainly make the conv layers of a convnet handle images of any size, without retraining. However, the output of a convnet will typically be some kind of classifier, and this will probably work less well, if you feed in inputs of different size, I would imagine.
Another approach would be to just pad the images with zeros. But imagine intuitively you are looking at either a tiny photo, padded with black borders, or you can zoom in, so it subtends a reasonable arc in your visual field. Which would you do? Which is easier to see?
