2
$\begingroup$

I'm very new to Machine Learning and I'm working on classifying some reviews into Good or Bad reviews. I have a training set of about 20,000 reviews.

I'm planning on using the "Bag of Words" approach. Now, I don't know whether I should use Word Count, Word Frequency or TFIDF?

My understanding is that most of the bad reviews will contain words like "terrible", "broke", "useless" and so shouldn't just TF be enough? What benefit will IDF bring to me?

$\endgroup$
3
  • $\begingroup$ Some useless words will have high TF and often they appear very often in other documents, so we need to downweight them by the IDF. $\endgroup$ Apr 13, 2020 at 4:27
  • $\begingroup$ @LernerZhang Yes, but what about words like "bad" and "broke" that may appear in all "Bad" reviews. That's a very good indicator whether a review is "Good" or "Bad", then won't TF-IDF reduce the weight of "bad" and "broke"? $\endgroup$ Apr 13, 2020 at 5:06
  • $\begingroup$ Hi @AlfroJang80, recall that TF is actually the proportion of the term in the document as opposed to the raw count. If we only use TF, the words "broke", "bad" etc. will only occupy a small proportion of the document. Although the signal is still there, it isn't very conducive to optimizing the classifier. If we use TF-IDF, although the raw values of "broke" and "bad" decrease, they will be more prominent relatively speaking. There could be other rare words such as the object in the review which have a higher value, however the classifier will not learn large weights for them. $\endgroup$ Apr 13, 2020 at 11:02

2 Answers 2

2
$\begingroup$

For sentiment analysis tasks like yours, you can either use "bag of words" or TF-IDF or "bag of vectors" or "bag of n-grams"(or others) to convert raw text to numerical features.

The Bag of Words and TF-IDF which is based on Bag of Words are very simple and fast algorithms. The IDF is necessary because it makes a trade-off between frequency of the term and how many documents include the term. The primary algorithm assumes that words, like stop words, that appear in other documents frequently are less valuable than those only appear in a document, resulting in it may not fit in your case where keywords would be counted as stop words.

You can utilize the "bag of n-grams" to prevent that, and you can also weight the n-grams by the Naive Bayes like that in this paper: Sentiment Classification using Document Embeddings trained with Cosine Similarity.

$\endgroup$
0
$\begingroup$

In TF we weighted terms based on the frequency of the word appearance in document. Some words that doesn't really have meaning may have more frequency count e.g. a, an, and, etc. More frequency count of a term may imply that the term is more important. IDF is a weight calculation method to prevent terms that have to many frequency count like stopwords from having high frequency count. IDF will work in a way that words that appear too often in a document will have lower weights and words that don't appear too often will have bigger weights.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.