Chi2 feature selection using large words classes in NLP

Question

As described here and there, Chi2 feature selection can be used in order to filter a set of words that have a low dependency with a given class C.

In my case, I have a corpus of documents where each is assigned a sentiment (positive or negative). After pre-processing my text (stopwords, stemming and removing words that appear in less than 1%/more than 50% of my documents), I define my positive class as any word contained in a document with positive sentiment, and same for the negative one.

I then use the Chi2 filtering approach with contingency tables. Below is an example of for word 'increase' and my positive class:

The problem I have is that my 2 classes are quite broad (hence the Chi2 filtering!) and there's always a word from each class in any document. Which makes my N01 and N00 equal to 0 for all the words.

The denominator of my Chi2 formula (below) is then systematically equal to 0 because of the 4th term.

Is there something that I'm not doing well? If not, is there a trick or an alternative I could use to avoid this problem?

Many thanks!

Answer 1

Chi-square assumes that you have a large enough expected value of each cell count $k \geq 5$. This means that each cell of the contingency table should have a minimal value of $5$.