I have a group of texts belonging to two different classes. I would like to extract numerical features that can separate well the two classes.

Right now I implemented a classic TF-IDF with a document for each text. This method may be not particularly useful in my opinion since it does not take into account the different classes but just looks at the important words in the texts.

Therefore I also implemented a TF-IDF fitted with a document for each class (so basically I combined all the texts of the same class together). I have never read about this, it is just my intuition, but I'm not sure it's correct.

I wonder if there are approaches similar to TF-IDF that extract the most important words for each document and also for each class.

Thank you!

  • $\begingroup$ Are you looking for something like LDA ("Latent Dirichlet Allocation")? $\endgroup$
    – cdalitz
    Feb 10, 2022 at 9:44

1 Answer 1


I would argue that your approach should not consider the class to which the document belongs.

Your approach would go something like this:

  1. Determine the class to which the document belongs.

  2. Do the class-specific feature extraction.

  3. Use the extracted features to predict the class to which the document belongs.

If you already know the class that lets you do class-specific feature extraction in step #2, you're done. You don't need to extract features. You know the answer with $100\%$ certainty. And if you don't know the class, then you can't do step #2.

For a concrete example, consider having news articles about politics and news articles about the Olympics. Based on the category to which each document belongs, you do category-specific feature extraction.

Along comes a new article, and you don't know if it is about politics or the Olympics. Do you do the feature extraction for political articles or articles about the Olympics?

  • $\begingroup$ Well in that example, I would compute the tf-idf on the new article, so that the words that appear a lot in politics texts and a little in news texts (and viceversa) have a higher value than the words which appear in both or never appear. My procedure is to fit the TF-IDF on just two documents with texts belonging to the two classes, and use that model to transform each text. $\endgroup$
    – inginging
    Feb 9, 2022 at 23:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.