Text Classification: training on long texts but application on short texts

Question

I have used a linear SVM model to perform a text classification. As I computed the TF-IDF features, which are based on the frequencies of term occurrence in documents, I'm worried about the effect of text size on the results.

My training corpus contained midsize text, and the model performs well. Now I would like to use it on short documents in which frequencies are probably less relevant. As a result, the model could be less reliable.

Is there a minimum text size under which it doesn't make sense to use the same model (this size depending on my training corpus)?

Or do I miss something and the TF-IDF inherently prevents such misbehaviour from the model?

Answer 1

There is no minimum text size on which to apply classifications. Though there is a rule of thumb: your training data should represent the data to which you're gonna apply your model. A model tested on small text-chunks will perform poorly or not at all if it was not trained on small text-chunks. There are variations of small/large depending on the application(s) at hand.

Your intuition is right however, the smaller the text-chunks and number of samples are, for either training or testing, the less relevant frequency is. Bottom line - the designation of appropriate text size is completely subjective and depends on the desired application. Validation will appropriately reflect the predictive quality of your classifications.