2
$\begingroup$

I am asking more of a theoretical question, I think. I'm doing some testing on two groups Group A (sample size 500) /Group B (sample Size 500) that are continuous ratio. They are both skewed distributions so I'm running a Wilcoxon Signed rank test. I'm getting significant results between the two (Z=26.928, pval=2.2e-16) and a large effect size (r= z/sqrt(N)=0.94).

Next I wanted to see whether I can predict Group A or B based on their data. I performed a logistic regression for this and found a 63% prediction rate for training (slightly lower for testing- 59%), which I am actually happy with because I do not want them to be predictable, but that's another story.

My question is if the groups are so different, why would the prediction be so low? I realize being different and being able to predict are two different things. I want to be sure my technique is Ok and if it is have some understanding as two why the groups could be so different, yet those differences cannot predict the group.

Improve this question
$\endgroup$
  • $\begingroup$ It sounds like your two groups are distinct subjects. If so you should be using a Wilcoxon test for independent groups, not a Wilcoxon signed rank test which is for paired measurements or two measurements made on the same subjects. $\endgroup$ – David Smith Jun 13 '17 at 18:49
  • $\begingroup$ Thank you. I did switch the test, but overall still get similar results that I cannot explain. $\endgroup$ – Tracy Jun 14 '17 at 13:57
0
$\begingroup$

You have a large sample that is fairly powerful.

The Wilcoxon test is a test for a difference in whether the cumulative distribution of Group A is to the left (or right) of Group B. When the distributions are symmetric, as in the Gaussian or logistic, then it is also a test of the equality of the location parameters of the distribution.

There can be a clear distinction in the two distributions but still considerable overlap between them. In that case, discrimination between them, predictability at each value, is more difficult even with large samples.

It sounds like the two samples are fairly close together but still distinguishable when you test the difference with a large sample.

Some data is just this way.

Improve this answer
$\endgroup$
  • $\begingroup$ Thank you, David. Your feedback is helpful. One thing I did was calculate the effect size wrong. II was dividing by sqrt(N/2) instead of sqrt(N) so my effect size is now .40, which makes much more sense than the 0.94 I originally got. $\endgroup$ – Tracy Jun 14 '17 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.