Best way to make an image classification model with dynamic image sizes

Question

I'm working on a project where I need an image classification system, so I've decided to learn Tensorflow, and, after a week of study i've the following model:

model = Sequential([
  layers.Rescaling(1./255, input_shape=(250, 250, 3)),
  layers.Conv2D(16, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(32, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(64, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Dropout(0.2),
  layers.Flatten(),
  layers.Dense(128, activation='relu'),
  layers.Dense(8)
])

as you probably have noticed is the model shown in the official tutorial in the Tensorflow's website.

However i've a problem, my users, can actually upload photo of different ratio (a 1:1 for avatar and 16:9/10:9 for thumbnail) but i don't want to make 3 (or more) different network for each ratio.

So what's best and why?

1. Simply scaling and deforming the image to match the size (and ratio) in the input shape
2. Change the input shape in 720p and add black strip to the smaller image and resize (with eventually black strip) the larger image
3. Read the image as 1D array and append to the end a bunch of zero until the input shape is filled

p.s. (If you can, if the best is the 3rd option, can you show me how to modify the model?)

Thank you for you time!

Answer 1

The solutions you suggest are valid except for the last one. Since you are using 2d CNN you cannot use 1d input.

In fact, your model already has a Rescaling preprocessing layer that will normalize the input to the range [0,1] and resize it to 250x250.

Scaling can negatively affect accuracy (especially upscaling), but it can be mitigated by training the model with data augmentation - cropping, rescaling, padding (adding black pixels) - whatever you will use at the inference step for your users.

Cropping may also cut out important parts of the image, so it depends on your task. For example, if it's face recognition, faces are usually centered, so cropping will work. You can also detect an object and crop it.

If you cannot retrain you model, I would choose to crop and downscale for large images and upscale to the full width/height for small ones