1
$\begingroup$

I have two groups of subjects (9 and 13 people). And data from two condition for this groups. In each condition for each subjects I have some measurements. But number of measurements are different for conditions (for second condition more then for first). First I use Wilcoxon-Mann-Whitney test for every subject to find subjects with significant difference between conditions. Second I use Fisher's Exact Test for understand is there significant difference in numbers of subjects with difference between condition for my two groups. Say, in first group 4 subjects show significant difference, and 3 in second. So my fisher's table is [[4,5],[3,10]]. Is this make any sense? and if so, Shall I use any correction for multiple comparisons?

$\endgroup$
4
  • $\begingroup$ Perhaps adding some more details and context would make this question clearer. For example, do you have repeated measures on each subject? How many Mann-Whitney tests did you do? $\endgroup$
    – Joel W.
    Oct 14, 2012 at 13:40
  • $\begingroup$ Yes I have repeated measures on each subject. But in second condition there are more measurements for each subject than in the first. I do 1 Mann-Whitney test for each subject. So it 9 for first group and 13 for second. $\endgroup$
    – sviter
    Oct 14, 2012 at 13:57
  • 1
    $\begingroup$ Can you tell us more about your research design? Is the following example more or less what you are doing? You have two groups, say men and women. You have two types of data for the men, say height and weight. You have the same two types of data for the women. You want to know which are heavier, men or women. You also want to know which are taller, men or women. Or is your design something different? $\endgroup$
    – Joel W.
    Oct 14, 2012 at 15:19
  • 1
    $\begingroup$ First of all, thank you for your attention. In term of your example, yes I have two groups - man and women. But I have one type of data, say blood pressure. I measured blood pressure n times for each subject. Then, say, they did physical exercises. And after it I again measured blood pressure for each subject, but m times. So m>n. I want to know in what group - man or women - chance of significant increase of blood pressure is more. $\endgroup$
    – sviter
    Oct 14, 2012 at 15:56

3 Answers 3

2
$\begingroup$

The two-stage approach seems to be "double-dipping" and is likely to result in an overall highly inflated type I error.

$\endgroup$
1
$\begingroup$

I assume you measured blood pressure (BP) multiple times before the physical exercise because you thought the BP measure is somewhat variable. If so, you could average all the pre-exercise BP measures and get one more reliable value for each person. As to the post-exercise measures of BP, you might do the same thing. If you think the post-exercise measures of BP will slowly return to baseline, you might average the first few post BP measures for each person to get one, more reliable post-exercise BP value for each person. At that point you will have one pre-exercise BP value (i.e., the before average) and one post-exercise BP value (i.e., the after average) for each person. Then you can analyze the data in various ways. One simple but effective approach is to calculate a before-after difference score for each person. Then you can do a Mann-Whitney test between men and women on these difference scores. (You could also consider using a t-test because it might have higher power, especially with your sample size.) Good luck.

$\endgroup$
4
  • $\begingroup$ Thanks, but I already did so. What I am interesting is MU+Fisher approach. Is this method make any sense? $\endgroup$
    – sviter
    Oct 15, 2012 at 8:24
  • 1
    $\begingroup$ What are you hoping to learn with that approach? What is your research question? $\endgroup$
    – Joel W.
    Oct 15, 2012 at 10:03
  • 1
    $\begingroup$ My research question is: how BP is changed and is any difference in this change between groups. But I want to use all data not just average. Because this approach yielded interesting, and slightly different results, and I try to understend is this method make any sense, or it is just an accident. $\endgroup$
    – sviter
    Oct 15, 2012 at 10:23
  • 2
    $\begingroup$ If you have a reason for not wanting to use average before and after scores, what is that reason? Do you think you will get more power by using a Fisher Exact Test to detect higher rate of significant multiple Mann-Whitney tests? I think it is likely that such a two stage approach has lower power than just doing a t-test on the before and after averages. In any case, if you want to "use all the data" have you considered doing an AVOVA with subjects as blocks? $\endgroup$
    – Joel W.
    Oct 15, 2012 at 12:17
0
$\begingroup$

If you're getting different answers with your approach versus a more traditional ANOVA, then it's probably because each subject brings their own variability to the BP measurements. ANOVA assumes homogeneous variances and lost information in your case. Unfortunately, the MU-FE approach is conditional upon the alpha-level you're using to test the conditional difference; if you had compared the before-after using alpha=0.25, would you get the same FE result? Your design appears to be a split-plot (whole plot=sex, subplot=before/after) aka repeated measures. And you are interested in testing to see if there is a significant interaction between the two factors. In this case, you maximize power by using all of the information. The suggestion of using the MU on the differences is a good step, but it assumes the differences across the subjects are measured with the same accuracy. Consider using a weighted t-test on the differences, where the weights are the sums of some (robust) measure of the before and after BP sample variabilities. If the number of measurements is the same across subjects, then just adding these measures of variability will incorporate the difference in number of observations between before and after. As long as the populations of differences are not too skewed, the t-test should provide a reasonably robust test.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.