How to do machine learning (regression/classification) when the samples are of different sizes?

Question

In standard cookbook machine learning, we operate on a rectangular matrix; that is, all of our data points have the same number of features. How do we cope with situations in which all of our data points have different numbers of features? For example if we want to do visual classification but all of our pictures have different numbers of pixels, or if we want to do sentiment analysis but all of our sentences have different amounts of words, etc.

I think the normal way would be to extract features of regular size from these irregularly sized data. But I attended a talk on deep learning recently where the speaker emphasized that instead of hand-crafting features from data, deep learners are able to learn the appropriate features themselves. But how do we use e.g. a neural network if the input layer is not of a fixed size?

Answer 1

I think this must be solved on a case-by-case basis. As for your first example, document analysis, probably based on bag-of-words, then you simply take the union of the two bags and introduce zero counts.

As for the image example, one commenter ask for scaling up the images with lowest resolution. Maybe that is the only solution, but then you will have some new pixels with values which are correlated with correlation equal to one! and that must be accounted for in the model.

Answer 2

I would try normalizing the data size using whatever algorithm provides the best results for your application. So if you're using images of variable resolution, before entering the image into the network you could scale or crop the image to the input size of the network.