As described here and here, $\chi^2$ feature selection can be used in order to filter a set of words that have a low dependency with a given class C.

In my case, I have a corpus of documents where each is assigned a sentiment (positive or negative). After pre-processing my text—stopwords, stemming and removing words that appear in less than 1% or in more than 50% of my documents—I define my positive class as any word contained in a document with positive sentiment and similarly for the negative one.

I then use the $\chi^2$ filtering approach with contingency tables. Below is an example of for word 'increase' and my positive class:

enter image description here

The problem I have is that my 2 classes are quite broad (hence the $\chi^2$ filtering!) and there's always a word from each class in any document. Which makes my $N_{01}$ and $N_{00}$ equal to 0 for all the words.

The denominator of my $\chi^2$ formula (below) is then systematically equal to 0 because of the 4th term.

$$ \chi^2(D, t, c) = \frac{N\left(N_{11}N_{00} - N_{10}N_{01}\right)^2} {(N_{11}+N_{01})(N_{11}+N_{10})(N_{10}+N_{00})(N_{01}+N_{00})}[4] $$

Is there something that I'm not doing well? If not, is there a trick or an alternative I could use to avoid this problem?

Many thanks!


1 Answer 1


Chi-square assumes that you have a large enough expected value of each cell count $k \geq 5$. This means that each cell of the contingency table should have a minimal value of $5$.

  • 3
    $\begingroup$ Welcome to Stats.SE. I've edited your answer to fix the typo (there is an edit button below your answer) - it should appear in a few minutes (after that, you may delete your comment since it's no longer needed). By the way, take the opportunity to take the tour( stats.stackexchange.com/tour), if you haven't done it already. See also some tips on formatting help and write down equations using LaTeX / MathJax. $\endgroup$ Commented Jul 10, 2019 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.