-1
$\begingroup$

I ran into this question at my lab when I was trying to transform my data using things like log transformation, square root etc. when my lab members just directly asked me to apply mann whitney test and be done with it since it doesn't matter if my data is normal or not in Mann-Whitney-Wilcoxon test.

I feel there is something seriously wrong going on. But I don't know what exactly. Is the above assumption taken by my lab members correct? If yes, what is all the fuss about techniques of transforming the data when I can blindly apply non-parametric tests?

$\endgroup$
6

1 Answer 1

0
$\begingroup$

For two-sample lognormal data, let's take a look at the results in R of three tests (a) Welch t test, (b) Wilcoxon rank sum test, and (c) Welch t test on logged (transformed) data.

We begin by sampling fictitious data in R and showing brief data summaries. [As usual the parameters of the lognormal distribution are those of the logged distribution.]

set.seed(1029)
x1 = rlnorm(20, 100, 13)
x2 = rlnorm(25, 113, 13)

summary(x1); length(x1);  sd(x1)
     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
1.619e+34 5.139e+38 1.494e+43 5.636e+49 3.696e+46 1.124e+51 
[1] 20             # sample size
[1] 2.512055e+50   # sample SD
summary(x2); length(x2);  sd(x2)
     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
1.963e+39 3.081e+44 1.910e+49 4.193e+57 1.475e+53 9.602e+58 
[1] 25
[1] 1.919156e+58

summary(log(x1)); sd(log(x1))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  78.77   87.91   97.35   96.77  106.84  117.55 
[1] 11.82917
summary(log(x2)); sd(log(x2))
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  90.48  102.44  113.47  112.75  122.43  135.81 
[1] 13.02913

The Welch test on lognormal data finds no significant difference.

t.test(x1, x2)

        Welch Two Sample t-test

data:  x1 and x2
t = -1.0923, df = 24, p-value = 0.2855
alternative hypothesis: 
 true difference in means is not equal to 0
95 percent confidence interval:
 -1.211453e+58  3.729247e+57
sample estimates:
   mean of x    mean of y 
5.636360e+49 4.192642e+57 

The 2-sample Wilcoxon test finds a highly significant difference in locations. If we judge the two sample to be of 'similar shape' we might say it's a difference in medians. If not, then x2 dominates x1.

wilcox.test(x1, x2)

        Wilcoxon rank sum test

data:  x1 and x2
W = 91, p-value = 0.0001614
alternative hypothesis: 
 true location shift is not equal to 0

The welch test on log-transformed data (which are normal) shows a significant difference. In terms of the original data we might interpret this as a ratio of geometric means that is not unity.

t.test(log(x1), log(x2))

        Welch Two Sample t-test

data:  log(x1) and log(x2)
t = -4.3032, df = 42.262, p-value = 9.764e-05
alternative hypothesis: 
 true difference in means is not equal to 0
95 percent confidence interval:
 -23.470083  -8.486333
sample estimates:
mean of x mean of y 
 96.77082 112.74903 

Simulation of 100,000 sets of such data show that the Welch t test on transformed data has slightly better power to distinguish between the two samples than does the Wilcoxon test.

Of course, results may differ for highly right-skewed data other than lognormal.

mu1 = 100;  mu2 = 113; sg = 13

# T TEST ON LOGNORMAL DATA
set.seed(1234)
pv.l = replicate(10^5, t.test(rlnorm(20,mu1,sg),
                  rlnorm(25,mu2,sg))$p.val)
mean(pv.l <= .05)
[1] 0.0012         # approximate power (nearly 0)

# WICOXON TEST ON LOGNORMAL DATA
set.seed(1234)
pv.w = replicate(10^5, wilcox.test(rlnorm(20,mu1,sg),
                    rlnorm(25,mu2,sg))$p.val)
mean(pv.w <= .05)
[1] 0.88525        # approximate power

# T TEST ON LOG-TRANSFORMED DATA (NORMAL)
set.seed(1234)
pv.n = replicate(10^5, t.test(log(rlnorm(20,mu1,sg)),
                        log(rlnorm(25,mu2,sg)))$p.val)
mean(pv.n <= .05)
[1] 0.90322       # approx power (slightly larger than Wilcoxon)

Both 'cures' for highly skewed non-normal data require interpretation as to what is being compared. The choice between a Wilcoxon test on lognormal data and a Welch t test on transformed data may depend on which interpretation seems more natural for the situation at hand.

$\endgroup$
2
  • 2
    $\begingroup$ 'Normalizing' is a poor choice of words. You should be calling this 'transforming'. And if you don't know more apriori, using a semiparametric regression model or its special case (a nonparametric test) is a robust and powerful approach. $\endgroup$ Commented Oct 29, 2021 at 13:07
  • $\begingroup$ @FrankHarrell, Edited Q&A to get rid of 'normalizing'. $\endgroup$
    – BruceET
    Commented Oct 29, 2021 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.