I have retrospectively collected clinical data of two sets of patients, one set with the diagnosis of tumor A (group A) and the other with tumor B (group B). There're 90 patients in group A, and 1100 patients in group B (as disease A is comparatively rare). My objective is to compare preoperative blood-test results of the two groups, to investigate if any blood-test markers could help differentiate the two groups preoperatively. I have a few questions regarding the study design to ask:

  1. I plan to first use Mann-Whitney U test to asess the differences between the two groups of each blood-test marker, for markers with significant differences, I'll then use ROC analysis to analyse their diagnostic performance. Finally, I plan to use logistic regression and/or SVM to build prediction models using the markers with good AUC values. I don't know if this plan is a good one or not, or any suggestions to improve it?

  2. Sex and age distributed differently between the two groups, as they are potential predictive factors of the two tumor types, I think I should control for them. I plan to match each patient in group A with 3 patients in group B (90:270, controled for sex and age). Is the design appropriate?

  3. I have performed preliminary/explortary tests on the data using python. I randomly sampled 270 patients from group B (controled for age and sex); then I looped the resampling precess hundres of times; so in the end, I have hundreds of samples (each sample with 90+450 patients) from the initial population (90+1100 patients in total). It turned out that none the the samples resulted in satisfactory AUCs, most AUCs were less than 0.7. If I want to report this negative result, what resampling/downsampling methods should I use in the formal research paper? My understanding is that resampling many times would give more convincing evidence that the assessed markers are not useful, but I doubt if the way I adopted is justifiable statistically or not? Or if it is unnecessary (i.e., result from a simple sample would be enough to draw the conclusion)?

I am pretty new to statistics, and the above question really confused me, I've searched for related questions and got no clear answers yet. Any advice would be appreciated! Thank you in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.