I leant that we can use chi-squared test for feature selection, especially this is useful on categorial features.

But as we need to create the contingency table and so forth. I am not sure if I can still apply this method on numerical response.

I've seen similar practices on chi squared for continuous features, the post saying that we can discretize the numerical feature into categorical one. I wonder if we can and we should use the same trick when it comes to numerical response.

Many thanks for your insightful feedbacks.

  • $\begingroup$ Question is too general for a targeted response. Without seeing some of your data and knowing your objectives, I wouldn't try to give advice. // Generally speaking, I would try not to reduce numerical data to ordinal categorical data in order to use a chi-squared test. It can be done, but often should not be done. // In reducing numerical data to bins you lose some information, then by ignoring the order of the bins and doing a chi-squared test, you lose still more information. // Even if a chi-squared test might be a simple way to start, you may find that's not the place to stop. $\endgroup$
    – BruceET
    Mar 22, 2022 at 3:13
  • $\begingroup$ @BruceET Thanks for your response. I am sorry that I didn't make my point more clearer. The whole purpose is just about what methods are available for us when we want to filter categorical features in general cases. Feature importance from tree based model is one, and I thought Chi-squared test might work too. But what I knew so far for chi use cases, are all categorical feature to categorical target variable test. That's why I wanna confirm my assumption that it can extend to numerical target variables. $\endgroup$
    – Tom
    Mar 22, 2022 at 15:43
  • $\begingroup$ But anyhow, you really make good points about the cons for numerical target case, like it goes through two phases of information loss, one at the time of binning the numbers, two at the time of doing chi-squared test, it ignores the ordinal nature in these bins, again. Thanks for your feedbacks. I think there are relative few methods on categorical feature selection. Maybe Lasso regression is one, and trick like PCA is another. $\endgroup$
    – Tom
    Mar 22, 2022 at 15:49


Browse other questions tagged or ask your own question.