Here I am asking about what others commonly do to use chi squared test for feature selection wrt outcome in supervised learning. If I understand correctly, do they test the independence between each feature and the outcome, and compare the p values between the tests for each feature?
Pearson's chi-squared test is a statistical test applied to sets of categorical data to evaluate how likely it is that any observed difference between the sets arose by chance.
A test of independence assesses whether paired observations on two variables, expressed in a contingency table, are independent of each other (e.g. polling responses from people of different nationalities to see if one's nationality is related to the response).
So must the two variables whose independence is tested by the test be categorical, or discrete (allowing ordered besides categorical), but not contiuous?
perform a $\chi^2$ test to the iris dataset to retrieve only the two best features.
In the iris dataset, all the features are numerical and continuous valued, and the outcome is class labels (categorical). How does the chi squared independence test apply to continuous features?
To apply chi squared independence test to the dataset, do we first convert the continuous features into discrete features, by binning (i.e. first discretizing the features' continuous domains into bins, and then replacing the features with occurrences of the features' values in the bins)?
Occurrences in several bins form a multinomial feature (either occur or not in each bin), so chi squared independence test can apply to them, right?
By the way I guess, can we apply chi squared independence test to features and outcomes of any kind, correct?
For the outcome part, we can select features for not only classification, but also for regression, by chi square independence test, by binning the continuous outcome, right?
The scikit learn site also says
Compute chi-squared stats between each non-negative feature and class.
This score can be used to select the n_features features with the highest values for the test chi-squared statistic from X, which must contain only non-negative features such as booleans or frequencies (e.g., term counts in document classification), relative to the classes.
Why does the test require nonnegative features?
If the features don't have signs but are categorical or discrete, can the test still apply to them? (See my part 1)
If the features are negative, we can always bin their domains and replace them with their occurrences (just like what I guess for applying the test to the iris dataset, see part 2), right?
Note: I guess Scikit Learn follows general principles, and that is what I am asking for here. If not, then it is still all right.