How to compare $t$-tests to $\chi^2$-tests for feature selection I am trying to learn feature importance for an ML model. To do this I am running a univariate statistical test between each independent feature and the dependent variable I am trying to predict. 
The dependent variable is binary and the features are mixed (some are categorical some are continuous). As a result, for the features that are continuous, I am running a two sample $t$-test and for the categorical features I am running a $\chi^2$-test. 
I want to be able to compare the outputs of these two different tests in order to see which features are the most closely related to the dependent variable. How do I compare the results from these two different tests? 
I am not sure how to do this as the test statistics are obviously different and just because the test shows that the relationship is statistically significant, this of course does not mean that the relationship is a particularly important one. For example, the mean difference between Group A and Group B could be very  statistically significant but only be a mean difference of a mere 0.01 units.
Therefore, how do I find the features that not only have the largest magnitude relationship with the dependent variable I am trying to predict but also the highest statistical significance. 
Thanks in advance for any help and let me know if I should provide more of my particular example.  
I am using scikit learn's feature_selection f_classif, chi2 methods to do this analysis. (They do what I previously described. two sample $t$-test and $\chi^2$-tests, respectively.)
 A: Regardless of whether you could, I don't think that you should. There are three problems with this approach:


*

*Significance does not correlate with predictive power. For example, two sample means may differ significantly while those samples' probability distributions overlap considerably. In this case, there is no separation that will yield a high prediction accuracy.

*The $t$-test compares the means of the groups, in relation to their standard error. The $\chi^2$-test compares the observed proportions of categories in either group to their expected proportions (expected if there were no difference). The $p$-values of these tests are evidence against different null-hypotheses.

*Even if you were to use another way to find the 'best' predictors, you would be using the data twice: Once to select variables and once to estimate the parameters of a model with those variables. This is a form of stepwise regression (which is a bad idea). 


Lastly, I don't see any mention of training/validation and where you would apply these tests. A significant difference in the groups in your sample need not imply an actual difference in those group's populations. However, just like the problem of multiple testing you mentioned in the comments, solving this will not solve the fundamental problem with this kind of variable selection.
So how then should you proceed? As mentioned in the comments and answers of this question, you may want to opt for a regularized regression model instead, which will shrink smaller coefficients to(wards) zero. Moreover, certain models can 'ignore' irrelevant predictors reasonably well, such as tree-based regression with random forests. Entire books have been written on model selection alone, but that Q&A is a very good starting point to better understand why this simply won't work well.
