If I have a multiclass perceptron that I'm using for document classification, and the dataset I'm using has different numbers of documents for each class, is there any normalization that has to be done to account for this? Because right now the model seems to be overfitting to the more common classes, which I suppose makes a little sense from a logical perspective because the a priori probability would be higher anyways, but it doesn't seem like an ideal outcome.


Normalization is always needed for the perceptron. In your case maybe you could try to preprocess further the data by having approximately the same number of documents per class.

Obviously if you have two classes with hundreds of docuw each and other two classes with just a couple documents your model will not even consider those classes.

Apart fron that, if you are using different sets of data for training and test, be sure that the percentage of documents of each class in each set is similar. This can be done by making a set per class and then creating the test and training set for each class, finally getting all the training sets together and all the test sets together and randomizing at least the final training set.

  • $\begingroup$ Did yoy get better results with any of these changes? $\endgroup$ – davidivad Nov 14 '15 at 2:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.