I am relatively new to Machine Learning and would appreciate your insights. I need to classify a set of keywords into five categories. I thought about using linear SVM for the exercise. I have a training dataset that contains the keywords and the classification and a second dataset that needs to be classified and only contains the keywords. What I know is that the unconditional distribution of the five categories is different in the training data than it is in the data that needs to be classified. How does that impact my learner?
In general a classifier (like a one-vs-all svm classifier) is trying to learn one function for each category. That function is supposed to receive an input of a set of keywords, as you describe it, and output a score, which is correlated to how likely the input set of keywords belongs to a specific category.
Every example that you give the classifier that is part of the category, lets the classifier know what kind of input belongs to the category. Every example which is labeled as not part of the category lets the classifier know what kind if input doesn't belong. The more different kinds of data that you give the classifier the better.
Theoretically, if you have no noise in your data and you give it all possible inputs, then your classifier should know how to classify perfectly. So a general rule of thumb is that the more of different kinds of data you give it the better. On the other hand, you classifier can't do everything and you don't want to overfit. if you're using a linear svm then your classifier can only draw straight lines in the input space and make decisions based on those lines. If 95% of the 'real' data that is part of category 1 is in region A and 5% of the real data that is in category 1 is in region B, and assume due to the choice of classifier, your classifier can only classify one of the regions as category 1, then you want it to find region A and ignore region B. You will have to live with 5% of category 1 data to be misclassified.
So this is where the proportion of data does matter. The more examples from region A you give it, the more likely it is to choose region A as its classify as category 1 region. The more examples from region B you give it, the more likely it is to choose region B as the area that if your input is there, it will classify it as category 1.
So what I recommend you do, is keep this in mind, do some data exploration to see how the data behaves (answering how to do this is a different question), choose a method (you said you were going to use SVM, I assume you have good reason to do so), test your performance on real data and see if it is acceptable. I'm not able to say exactly what you should do because it depends on the data, and from the behavior of the data you should choose what data preprocessing techniques and classifiers to use.
Sorry if my explanation of regions is confusing. I'm not good at drawing things, although it probably would have helped my explanation, so if someone can help or do a better job, please do.