I'm training a non linear svm to do classification on images. I'm featurizing the image by creating 3 features for each pixel, its rgb value. My question is: How should i normalize images of different dimensions. My initial thought is to go through all the images and find the maximum width and height; then extend all the images to that width and height and fill its missing pixel values with -1's. Will a non-linear svm work ok with this image normalization? How do other people deal with images of different sizes?
I would be more worried about having images of different aspect ratios than images of different sizes. The way I would approach the problem would be to discard some of the images (the one who are abnormally small) and downscale all the other to a common size. Why I did propose downsampling instead of upsampling the images? By upsampling you will be introducing a lot of noise and opportunity to overfit (high variance), downsampling will work the other way around... only on massive datasets do big images appear.
A personal recommendation: why use svm for image classification? By my experience you would be much better suited with a CNN classifier, it is better suited to capture the non-linearities that are common on this kind of tasks and they overfit much less, when used wisely of course.