0
$\begingroup$

I want to compare the overall mean weight between males and females in my biological sample. The weight is not normally distributed so I have used a Wilcoxan ranked sum test and found significant differences in weight between sexes. In this dataset, I have roughly 550 females and 350 males.

I also have a separate dataset where samples were taken using a separate sampling technique and I compared the exact same thing; weight between each sex and found no significant between sex but I had roughly 150 females and 50 males.

I want to compare results between sampling techniques but I'm cautious because of the differences in sample size between samples and between sex.

In an effort to correct for the lower sample size in the second method, I randomly sampled the same number of males and females from the first dataset and completed the same analysis 1000 times and found that ~600-800 times I found no significant differences in weight between sex.

$\endgroup$
1
  • $\begingroup$ You are not going to have the same power to detect a M/F difference with the reduced number of observations. In terms of power, the most efficient design would be to have roughly equal sample sizes for both M and F. Having sample sizes 150 & 50 provides more info than 50 & 50, but not much more. "A chain is not stronger than it's weakest link." // Also, also huge re-sampled collections from a small dataset have no more information than the original small dataset. Re-sampling has its place in data analysis, but not in mysteriously creating info about the real world. $\endgroup$
    – BruceET
    Mar 21, 2021 at 21:26

1 Answer 1

0
$\begingroup$

You don't say anything about the actual weights you are dealing with. So I'll choose some hypothetical values for illustration. Suppose women have average weights about 230 (pounds) with standard deviation 45, and men averaging about 245 with SD 55.

Then respective sample sizes 150 and 50 may not be enough to detect the 15 lbs difference in gender weights. Then according to the following simulation in R, a little over half of the experiments according to these parameters would show a significant difference. (The power is the probability of detecting such a difference).

set.seed(2021)
pv = replicate(10^4, 
      wilcox.test(rnorm(150, 230, 45), 
                  rnorm(50, 245, 55), alt="l")$p.val)
mean(pv <+ .05)
[1] 0.5502    # power of Wilcoxon rank sum test

However, with the same weight distributions and sample sizes 550 and 350, you would almost certainly detect the difference in gender weights.

set.seed(321)
pv = replicate(10^4, 
      wilcox.test(rnorm(550, 230, 45), 
                  rnorm(350, 245, 55), alt="l")$p.val)
mean(pv <+ .05)
[1] 0.9943

Note: There are procedures in most statistical software (and online) for the power of t tests (requiring normal data) where both samples are the same size. If sample sizes are different, then one can use formulas involving noncentral t distributions to compute power. However, I know of no such procedures for Wilcoxon signed rank tests. Thus, I have used simulation for this example.

$\endgroup$
4
  • $\begingroup$ In 2nd dataset is it 150 (as in Comment above) or 50 (as in original Q) Males? $\endgroup$
    – BruceET
    Mar 22, 2021 at 2:39
  • $\begingroup$ error on my part - should be 50 for males in the second set. I will edit my post $\endgroup$ Mar 22, 2021 at 2:41
  • $\begingroup$ Attached is the data to provide context. The first Dataset; Male (n=350) mean=1.68 kg. with sd = 0.49. Female (n=550) mean=1.76 kg. with sd =0.46. The second dataset; Male (n=50) mean=1.63 kg. with sd=0.41. Female (n=150) mean = 1.72 kg. with sd=0.45. I entered my values into your simulation and had odd results. For dataset one with larger samples, a value=0 was given as the power multiple times. The second dataset had a slightly higher value of 0.0013. The way I Interpret that result states that the lower sample size dataset contains more power but hardly any power at all? $\endgroup$ Mar 22, 2021 at 2:42
  • 1
    $\begingroup$ I reviewed the code and I placed the larger female means first and left the L in the alt argument therefore it explains why I was having issues. $\endgroup$ Mar 22, 2021 at 2:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.