11
$\begingroup$

I have a dataset of wide images: 1760x128. I've read though tutorials and books, and most of them state that input images should be square and if not, they are transformed to square in order to be trained in already trained (on square images) cnns. Is there a way to train cnn for non square images, or should I look for another option as padding?

$\endgroup$
3
$\begingroup$

There are several ways to solve the problem depending on the classifier. Sliding Windows is the method I'm most familiar with, this is used for the neural network methods. This method involves taking a small sub-image and shifting it up and down with some overlaps. Some issues include finding the optimum shift parameters and multi scale-issues.

The final detection is usually determined by how confident the classifier is that each of the sub-images belong in that class: for example majority vote, total likelihood or total distance from the decision boundary. I have listed some material below, the first one is for the HOG classifier method but the concepts are the same.

  1. Object Detection Sliding Windows
  2. Object Category Detection: Sliding Windows
  3. OverFeat Integrated Recognition, Localization and Detection using Convolutional Networks
| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

This shouldn't cause any problems at all if you are using a CNN. I made a CNN for recognizing faces, and since faces are usually around 70% as wide as they are tall, I used training images that are 80x100 pixels (a little extra width in case the head was at an angle). Your filters should still be squares though.

All that changes would be that now you have to keep track of a width and a height for your activation/pooled maps instead of just one value that tells you the size. For example -

Input image of 80 x 100 Apply 5 x 5 convolution filter gives a map of activations at 76 x 96 Apply 2 x 2 pooling gives a map of pooled activations at 38 x 48

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

Just stick with squares, it's safe & easy

Most CNN training datasets have images that are not square. The standard method is to take a square crop out of it -- often picking a random square for training, and at test time to use multiple squares and aggregate the predictions (center + 4 corners is a classic).

Cropping is preferable to padding because with padding you're wasting a lot of compute on blank image space, and you're creating this artificial edge where the real picture has a hard line against a solid color, which can confuse the classifier. So the easiest thing is to just stick with common practice and crop to square.

This is general advice, which works well for typical camera images that have moderate aspect ratios like 1.5:1. In your case where the aspect ratio is extreme, cropping is obviously going to greatly limit what the network can physically see.

Using different sizes

That said most modern CNN's do allow you to run non-square images through them. This is somewhat advanced / unusual, so better to stick to square images unless you're confident you understand what's going on and the risks involved. Whether or not this is physically possible for a type of CNN is determined by the network layer near the end, that transitions from the convolutional processing to the fully-connected processing for classification or what-have-you.

In early CNN's like AlexNet, this layer just serialized all the channel maps from the reduced-resolution processed image into a simple vector of features - so if your last conv layer outputs an 8x8x32 Tensor (HxWxChannel) this just unrolls it to a 2048 dimensional vector. This strategy encodes each position of the image into a different set of dimensions in the vector, allowing the network to reason about features at different locations of the image (can be good or bad), but also means that the resolution it can work with is fixed, because if you change the resolution, the dimensionality of the feature vector would change, and then the down-stream fully-connected layers just wouldn't work.

Most modern CNN's instead use some kind of pooling layer to average or aggregate the location-based features across the spatial dimensions into a simple feature vector (sometimes accurately described as 1x1 resolution) for the down-stream FC layers. So if that same network that output 8x8x32 features reached this pooling layer, it might average all 64 (=8x8) of those 32-dim vectors into a single 32-dimensional feature vector for downstream processing. (In practice these nets have more than 32 features at the end.)

With these networks, there's nothing physically stopping you from running different sized images through the network. Some good details in e.g. this answer: https://stats.stackexchange.com/a/392854/13947 But always remember that if you ask a network to do anything it wasn't trained to do, it's extremely likely to behave badly!

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.