Say I have a total population of N=100. Algorithm #1 is based on some predictors and it selects n1=10 subjects (i.e., subgroup #1). Algorithm #2 is based on another set of predictors and it selects n2=50 subjects (i.e., subgroup #2) from the same original population (N=100).

Some of these selected subjects appear twice in both Algo #1 and Algo #2 derived subgroups, while each subgroup also has its own unique subjects.

Now, each subgroup has an average age, am I correct to use paired t-test to test for statistical difference? Or should I use a different test?



What I want to test for is whether the mean age of subgroup #1 is statistically different (at p=0.05) than the mean age of subgroup #2 (considering the fact that some subjects appear in both subgroups, while others only appear in either subgroup #1 or subgroup #2).

  • $\begingroup$ N=100, N=10, N=50, N=100. We have 26 letters. $\endgroup$ – user158565 Oct 30 '18 at 17:48
  • $\begingroup$ What are you trying to test for? Your question seems to be missing the detail of what you want to know from or about the samples or population or algorithms. $\endgroup$ – Michael Lew Oct 30 '18 at 19:32

I suggest you partition your data into those selected by both algorithms and those not selected by both algorithms.

Let's consider those subsets in turn.

Obviously the ones that appear in both groups will have the same age as themselves; their pair-differences can contribute no information about age difference.

The remaining subset do contain information about age differences; they are unpaired so you should not use a paired test for those.

  • $\begingroup$ Thanks, but the goal is not to find people signaled by both algorithms. These are two case definitions to identify patients, and we don't know which case definition is better at the moment, we do want to see how statistically different some of the characteristics are of the subgroups identified by these case definitions. So if they are similar (not statistically different) and the overlap between them is large, then we can say these case definitions perform similarly. $\endgroup$ – KubiK888 Oct 30 '18 at 22:26
  • $\begingroup$ Thanks for your suggestion. It means good logical sense. $\endgroup$ – KubiK888 Oct 31 '18 at 16:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.