1
$\begingroup$

I am wondering whether running multiple hypothesis tests (t-test / Mann Whitney) as a first step in classification problem.

Specifically: given a data set with k features (k=3 in the example bellow), one can run t-test k times, using each feature at a time, fine the p-value corresponding to whether the distribution of feature_i in class 0 is statistically different than in class 1. Correct for multiple comparisons and keep only features which p-values 'survived' the correction and are < 0.05. Then, these features will be used in further classification model.

enter image description here

In this example, I compute two-sample t-test (for class 0 and 1).

Does this procedure make sense at all or it makes more sense to use regularisation to find the best model (features)?

$\endgroup$

2 Answers 2

1
$\begingroup$

Just throw them all into a random forest. It's quite fast and robust to noisy predictors.

$\endgroup$
1
  • $\begingroup$ thanks! I'm actually doing XGBoost classification, however, I observed, that this kind of feature pre-selection my model performed better. $\endgroup$ Aug 11, 2019 at 22:36
0
$\begingroup$

This is not a good approach, because it does not consider any interactions between any of the variables, since a t-test is a univariate test, that is checking one variable at a time.

That's what more complex models are better at. Regularization would be a better option then multiple t-tests.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.