I am currently struggling to find the most proper test to serve my needs. The thing is, I have one data matrix. The columns are computational features and each row corresponds to a patient. The last column of the matrix is the label of the patient, which is called 'Outcome'. There are two categories: R (Responder to immuno-therapy) and NR (Non-responder to therapy). Now, I want to select the most informative features and then input to the support vector machine for patient classification. To achieve this goal, I did the followings:

First, for each feature, I split the data to NR-associated values and R-associated values, then I would like to perform a 'test' for these two vectors to see if there is a significant difference. If yes, the feature is maintained, if no, the feature is discarded.

Second, since there are thousands of features, therefore I need to perform the test for thousands of times. So after I calculated the p value, I also performed a Bonferroni correction to obtain adjusted p value.

The adjusted p values are used for feature screening.

Initially, I used Student's t-test. However, I am struggling if this is the appropriate test to fit the scenario here, since:

  1. There are only ~150 patients. The number of samples is not that high.
  2. The length of Responder's vector and Non-Responder's vector are not the same. For example, there are ~100 responders but only ~50 non-responders.

Can anyone please give me some suggestions and explain the rationale? Thank you!

  • $\begingroup$ I see no difficulty using a two-sample t test with sample sizes 50 and 100, provided that the data are nearly normal. I would use a Welch test instead of a pooled test unless it is somehow clear that the two groups must have equal variances. $\endgroup$
    – BruceET
    Aug 2, 2020 at 10:18
  • $\begingroup$ A t-test compares means while I don't see the word "mean" in your question... note that your approach will lead to a very biased classificator performance. $\endgroup$
    – Michael M
    Aug 2, 2020 at 11:41

1 Answer 1


As said by @BruceET, you can use the Welch test to identify the differential features among the two groups.

Be careful about overfitting. Feature selection performance can be evaluated by cross-validation, for example. In that case, you have to run the feature selection on the only training set.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.