How do I classify images with non-rectangle shape with CNN?

Question

How do I choose filter? Should I zero-padding the rest of the image and make it a rectangle? Or in a more specific case, while dealing with brain fMRI datas, the 3D models where brains are represented are not perfectly matched into cubes. This may makes the traditional convolutional neural network problematic as it makes us harder moving a filter in all offsets.

PS.

2D model (images) is just a simplification for my case. I am currently doing a brain fMRI classification model, though we can view the data in a 60*50*20 cube, but in fact, imagine the shape of the brain (the cube is just created for the ease of assigning coordinates or each voxel), over 50% of the cube has no data (no data means N/A, and not 0, they would not contribute anything to my model). So what should I do now?

Answer 1

You can perfectly train non-rectangular images with CNN. if you have non rectangular input image, you can still use the square kernels. Note that the resulting size will be different in two dimensions. If you have 80*100 image, and use 5*5 filter, resulting dimension will be 76*96 if you do not use any padding. You have to be careful with this dimension because - at the final layer you are probably going to use fully connected layer which size has to match exactly - otherwise you will get dimension mismatch error.

In your case, if you have some points missing, one way to use is to go with copy padding, which is similar to zero padding except it adds the values of border pixels to newly added dimensions. And if you think you have to add more more pixels by about 50%, it's better not to use that type of data to train-otherwise its going to add more noise to your data - and making your classifier more confusing.