8
$\begingroup$

Is chi-squared feature selection better than Mutual information based feature selection mechanism?

$\endgroup$
3
  • 3
    $\begingroup$ Better in what respect? $\endgroup$ Commented Nov 27, 2014 at 2:19
  • $\begingroup$ In classification accuracy when used with SVM. $\endgroup$
    – Ankit Das
    Commented Nov 27, 2014 at 6:06
  • $\begingroup$ I would say that MI is better based upon the premises on which the Chi-squared is developed. Also given that there is the 'vanishing p-value' problem for large datasets and the complexity of dealing with significance testing in those situations, that MI is going to be more robust for machine learning applications. $\endgroup$
    – Vass
    Commented Oct 4, 2018 at 14:53

2 Answers 2

7
$\begingroup$

They are related, so I don't suspect there to be a big difference (hence, go for mutual information if it's easier to calculate).

I haven't seen a formal argument for this, but my logic is:

  • A g-test is a derivate of mutual information ($G=2\cdot N \cdot MI(r,c)$, cfr. wiki link)
  • A Chi-squared leads to the same conclusion as a g-test for reasonably sized samples

Therefore, Chi-squared and MI lead to more or less the same results for reasonably sized samples. In other cases, it will deterministically depend on the dataset properties.

$\endgroup$
1
$\begingroup$

Just as a follow up to @ciri answer, the same arguments have been developed in the following paper: Richter et al., 2018

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.