Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a rather large collection of text documents categorized into about 150 categories. While some categories are represented by several thousands of documents, others have only a few hundreds assigned to them. Now I would like to construct a balanced corpus from this data, where each category is represented by the same number of documents and which at the same time maximizes the categorization accuracy. I tried to randomly select documents from the stronger categories but would like to know if there is a more systematic way.

share|improve this question

Some suggestions:

  1. Choose the minimum class size, M, select the M nearest instances to the class means.

  2. Find the maximum class size, N. Add Gaussian random noise to a random sample with replacement of N-Si instances for each class, other than the largest. S is the class size for the ith class.

  3. For each class, except the smallest, run k-means clustering. k is the number of instances in the smallest class. Use the cluster centers as the instances for the class.

It may be worth mentioning that with option 2 you want to take care not to let the generated additive nose samples leak into the test set for your evaluation.

share|improve this answer
Thanks four your suggestions. I tried number one and constructed a corpus by selecting the 100 documents with the smallest Euclidean distances from the class centroids (122 classes and 12200 documents in total). The categorization performance in terms of average F1 was 0.485 compared to 0.319 in the case of randomly selected documents. Next, I am going to try the other two. – mathias Jul 5 '12 at 6:34
@mathias I doubt if these approaches are optimal, but at least things are going in the right direction :) – image_doctor Jul 5 '12 at 6:55
@mathias Let us know how it goes, cheers :) – image_doctor Jul 5 '12 at 10:11

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.