Return to Answer

While $U$ is asymptotically normally distributed with a simple form for mean and variance ($\mu=n_1n_2/2, \sigma^2=\mu(n_1+n_2+1)/6$), standardizing U for the mean and s.d. of its own distribution seems to me a dubious measure of effect size (whether or not one subsequently standardizes for $N = n_1+n_2$ as well).

It's not clear what the effect size at that page you link to actually measures. They don't say what they think it represents. But in any case several things they say on that page are wrong. Given the errors there, I would be cautious of what is there unless it had a good justification.

To my mind, effect size should convey something meaningful about the difference between the groups - some meaningful quantity in terms of the original problem, such as:

• some estimate of the difference in population locations (possibly standardized, though personally I'd prefer it in the original units); or

• some estimate of the probability that a random X-value exceeds a random Y-value

(though other measures are possible).

$U$ can be seen as a shifted sum of ranks in the first sample, or a scaled and shifted difference in mean rank.

Casting effect size in terms of a shift in mean rank seems odd, because it doesn't tell us how much movement there is.

If you're willing to do that, you might as well just transform the p-value via an inverse normal cdf and be done with it, for all that tells you. If your sample sizes are big enough to use the normal approximation, they're already the same thing.

$U$ could be seen as a scaled form of the sample version of that $X>Y$ probability above, but the scaled version of the probability doesn't seem to give an intuitive interpretation. If a z-score computed from that kind of probability makes sense for your problem, I suppose you could do that. Once standardized, I have no real feel for how much effect that actually is though. Converted to a Z-score it should be the same as the Z-score mentioned in your post ($\frac{U-\mu_U}{\sigma_U}$).

If you're looking at location shift alternatives, there's already an estimate of the shift in location (the Hodges-Lehmann estimator, which is the median pairwise difference). That estimate and an interval for the population shift can both be generated relatively simply and many stats packages implement that already, as with this example from R:

> wilcox.test(x,y,conf.int=TRUE)

    Wilcoxon rank sum test

data:  x and y
W = 89, p-value = 0.02346
alternative hypothesis: true location shift is not equal to 0
95 percent confidence interval:
 -8.510 -0.689
sample estimates:
difference in location 
               -4.4715

In my example above, it gave an estimated location-shift of $-4.4715$ with a 95% interval for the shift $(-8.510, -0.689)$

The estimate of the probability that an X exceeds a Y is $U/n_1n_2 = 0.297$. [I don't know a suitable nonparametric way of generating an interval for that when it's different from $\frac{_1}{^2}$ though.]

As for the difficulty that you don't have the original $U$'s I don't know what can be done, but I'd suggest asking the original authors for the data... or at least the $U$'s.

While $U$ is asymptotically normally distributed with a simple form for mean and variance ($\mu=n_1n_2/2, \sigma^2=\mu(n_1+n_2+1)/6$), standardizing U for the mean and s.d. of its own distribution seems to me a dubious measure of effect size (whether or not one subsequently standardizes for $N = n_1+n_2$ as well).

It's not clear what the effect size at that page you link to actually measures. They don't say what they think it represents. But in any case several things they say on that page are wrong. Given the errors there, I would be cautious of what is there unless it had a good justification.

To my mind, effect size should convey something meaningful about the difference between the groups - some meaningful quantity in terms of the original problem, such as:

• some estimate of the difference in population locations (possibly standardized, though personally I'd prefer it in the original units); or

• some estimate of the probability that a random X-value exceeds a random Y-value

(though other measures are possible).

$U$ can be seen as a shifted sum of ranks in the first sample, or a scaled and shifted difference in mean rank.

Casting effect size in terms of a shift in mean rank seems odd, because it doesn't tell us how much movement there is.

If you're willing to do that, you might as well just transform the p-value via an inverse normal cdf and be done with it, for all that tells you. If your sample sizes are big enough to use the normal approximation, they're already the same thing.

$U$ could be seen as a scaled form of the sample version of that $X>Y$ probability above, but the scaled version of the probability doesn't seem to give an intuitive interpretation. If a z-score computed from that kind of probability makes sense for your problem, I suppose you could do that. Once standardized, I have no real feel for how much effect that actually is though. Converted to a Z-score it should be the same as the Z-score mentioned in your post ($\frac{U-\mu_U}{\sigma_U}$).

If you're looking at location shift alternatives, there's already an estimate of the shift in location (the Hodges-Lehmann estimator, which is the median pairwise difference -- note, however, that this median difference is not in general the same as the difference in medians). That estimate and an interval for the population shift can both be generated relatively simply and many stats packages implement that already, as with this example from R:

> wilcox.test(x,y,conf.int=TRUE)

    Wilcoxon rank sum test

data:  x and y
W = 89, p-value = 0.02346
alternative hypothesis: true location shift is not equal to 0
95 percent confidence interval:
 -8.510 -0.689
sample estimates:
difference in location 
               -4.4715

In my example above, it gave an estimated location-shift of $-4.4715$ with a 95% interval for the shift $(-8.510, -0.689)$

The estimate of the probability that an X exceeds a Y is $U/n_1n_2 = 0.297$. [I don't know a suitable nonparametric way of generating an interval for that when it's different from $\frac{_1}{^2}$ though.]

As for the difficulty that you don't have the original $U$'s I don't know what can be done, but I'd suggest asking the original authors for the data... or at least the $U$'s.

While $U$ is asymptotically normally distributed with a simple form for mean and variance ($\mu=n_1n_2/2, \sigma^2=\mu(n_1+n_2+1)/6$), standardizing U for the mean and s.d. of its own distribution seems to me a dubious measure of effect size (whether or not one subsequently standardizes for $N = n_1+n_2$ as well).

It's not clear what the effect size at that page actually measures. They don't say what they think it represents. But in any case several things they say on that page are wrong, so that may not be saying much. Given the errors there, I wouldn't rely on anything that page says unless it had a good justification.

To my mind, effect size should convey something meaningful about the difference between the groups - some meaningful quantity in terms of the original problem, such as:

• some estimate of the difference in population locations (possibly standardized, though personally I'd prefer it in the original units); or

• some estimate of the probability that a random X-value exceeds a random Y-value

(though other measures are possible).

$U$ can be seen as a shifted sum of ranks in the first sample, or a scaled and shifted difference in mean rank.

Casting effect size in terms of a shift in mean rank seems odd, because it doesn't tell us how much movement there is.

If you're willing to do that, you might as well just transform the p-value via an inverse normal cdf and be done with it, for all that tells you. If your sample sizes are big enough to use the normal approximation, they're already the same thing.

[$U$ could be seen as a scaled form of the sample version of that probability, but the scaled version of the probability doesn't seem to give an intuitive interpretation.]

If you're looking at location shift alternatives, there's already an estimate of the shift in location (the Hodges-Lehmann estimator, which is the median pairwise difference). That estimate and an interval for the population shift can both be generated relatively simply and many stats packages implement that already, as with this example from R:

> wilcox.test(x,y,conf.int=TRUE)

    Wilcoxon rank sum test

data:  x and y
W = 89, p-value = 0.02346
alternative hypothesis: true location shift is not equal to 0
95 percent confidence interval:
 -8.510 -0.689
sample estimates:
difference in location 
               -4.4715

In my example above, it gave an estimated location-shift of $-4.4715$ with a 95% interval for the shift $(-8.510, -0.689)$

The estimate of the probability that an X exceeds a Y is $U/n_1n_2 = 0.297$. [I don't know a suitable nonparametric way of generating an interval for that when it's different from $\frac{_1}{^2}$ though.]

As for the difficulty that you don't have the original $U$'s I don't know what can be done, but I'd suggest asking the original authors for the data... or at least the $U$'s.

While $U$ is asymptotically normally distributed with a simple form for mean and variance ($\mu=n_1n_2/2, \sigma^2=\mu(n_1+n_2+1)/6$), standardizing U for the mean and s.d. of its own distribution seems to me a dubious measure of effect size (whether or not one subsequently standardizes for $N = n_1+n_2$ as well).

It's not clear what the effect size at that page you link to actually measures. They don't say what they think it represents. But in any case several things they say on that page are wrong. Given the errors there, I would be cautious of what is there unless it had a good justification.

To my mind, effect size should convey something meaningful about the difference between the groups - some meaningful quantity in terms of the original problem, such as:

• some estimate of the difference in population locations (possibly standardized, though personally I'd prefer it in the original units); or

• some estimate of the probability that a random X-value exceeds a random Y-value

(though other measures are possible).

$U$ can be seen as a shifted sum of ranks in the first sample, or a scaled and shifted difference in mean rank.

Casting effect size in terms of a shift in mean rank seems odd, because it doesn't tell us how much movement there is.

If you're willing to do that, you might as well just transform the p-value via an inverse normal cdf and be done with it, for all that tells you. If your sample sizes are big enough to use the normal approximation, they're already the same thing.

$U$ could be seen as a scaled form of the sample version of that $X>Y$ probability above, but the scaled version of the probability doesn't seem to give an intuitive interpretation. If a z-score computed from that kind of probability makes sense for your problem, I suppose you could do that. Once standardized, I have no real feel for how much effect that actually is though. Converted to a Z-score it should be the same as the Z-score mentioned in your post ($\frac{U-\mu_U}{\sigma_U}$).

If you're looking at location shift alternatives, there's already an estimate of the shift in location (the Hodges-Lehmann estimator, which is the median pairwise difference). That estimate and an interval for the population shift can both be generated relatively simply and many stats packages implement that already, as with this example from R:

> wilcox.test(x,y,conf.int=TRUE)

    Wilcoxon rank sum test

data:  x and y
W = 89, p-value = 0.02346
alternative hypothesis: true location shift is not equal to 0
95 percent confidence interval:
 -8.510 -0.689
sample estimates:
difference in location 
               -4.4715

In my example above, it gave an estimated location-shift of $-4.4715$ with a 95% interval for the shift $(-8.510, -0.689)$

The estimate of the probability that an X exceeds a Y is $U/n_1n_2 = 0.297$. [I don't know a suitable nonparametric way of generating an interval for that when it's different from $\frac{_1}{^2}$ though.]

As for the difficulty that you don't have the original $U$'s I don't know what can be done, but I'd suggest asking the original authors for the data... or at least the $U$'s.

While $U$ is asymptotically normally distributed with a simple form for mean and variance ($\mu=n_1n_2/2, \sigma^2=\mu(n_1+n_2+1)/6$), standardizing U for the mean and s.d. of its own distribution seems to me a dubious measure of effect size (whether or not one subsequently standardizes for $N = n_1+n_2$ as well).

It's not clear what the effect size at that page actually measures. They don't say what they think it represents. But in any case several things they say on that page are wrong, so that may not be saying much. Given the errors there, I wouldn't rely on anything that page says unless it had a good justification.

To my mind, effect size should convey something meaningful about the difference between the groups - some meaningful quantity in terms of the original problem, such as:

• some estimate of the difference in population locations (possibly standardized, though for my money much better in the original units); or

• some estimate of the probability that a random X-value exceeds a random Y-value

(though other measures are possible).

$U$ can be seen as a shifted sum of ranks in the first sample, or a scaled and shifted difference in mean rank.

Casting effect size in terms of a shift in mean rank seems odd, because it doesn't tell us how much movement there is.

If you're willing to do that, you might as well just transform the p-value via an inverse normal cdf and be done with it, for all that tells you. If your sample sizes are big enough to use the normal approximation, they're already the same thing.

If you're looking at location shift alternatives, there's already an estimate of the shift in location (the Hodges-Lehmann estimator, which is the median pairwise difference). That estimate and an interval for the population shift can both be generated relatively simply and many stats packages implement that already:

> wilcox.test(x,y,conf.int=TRUE)

    Wilcoxon rank sum test

data:  x and y
W = 89, p-value = 0.02346
alternative hypothesis: true location shift is not equal to 0
95 percent confidence interval:
 -8.510 -0.689
sample estimates:
difference in location 
               -4.4715

In my example above, it gave an estimated location-shift of $-4.4715$ with a 95% interval for the shift $(-8.510, -0.689)$

The estimate of the probability that an X exceeds a Y is $U/n_1n_2 = 0.297$. [I don't know a suitable nonparametric way of generating an interval for that when it's different from $\frac{_1}{^2}$ though.]

As for the difficulty that you don't have the original $U$'s I don't know what can be done, but I'd suggest asking the original authors for the data... or at least the $U$'s.

While $U$ is asymptotically normally distributed with a simple form for mean and variance ($\mu=n_1n_2/2, \sigma^2=\mu(n_1+n_2+1)/6$), standardizing U for the mean and s.d. of its own distribution seems to me a dubious measure of effect size (whether or not one subsequently standardizes for $N = n_1+n_2$ as well).

It's not clear what the effect size at that page actually measures. They don't say what they think it represents. But in any case several things they say on that page are wrong, so that may not be saying much. Given the errors there, I wouldn't rely on anything that page says unless it had a good justification.

To my mind, effect size should convey something meaningful about the difference between the groups - some meaningful quantity in terms of the original problem, such as:

• some estimate of the difference in population locations (possibly standardized, though personally I'd prefer it in the original units); or

• some estimate of the probability that a random X-value exceeds a random Y-value

(though other measures are possible).

$U$ can be seen as a shifted sum of ranks in the first sample, or a scaled and shifted difference in mean rank.

Casting effect size in terms of a shift in mean rank seems odd, because it doesn't tell us how much movement there is.

If you're willing to do that, you might as well just transform the p-value via an inverse normal cdf and be done with it, for all that tells you. If your sample sizes are big enough to use the normal approximation, they're already the same thing.

[$U$ could be seen as a scaled form of the sample version of that probability, but the scaled version of the probability doesn't seem to give an intuitive interpretation.]

If you're looking at location shift alternatives, there's already an estimate of the shift in location (the Hodges-Lehmann estimator, which is the median pairwise difference). That estimate and an interval for the population shift can both be generated relatively simply and many stats packages implement that already, as with this example from R:

> wilcox.test(x,y,conf.int=TRUE)

    Wilcoxon rank sum test

data:  x and y
W = 89, p-value = 0.02346
alternative hypothesis: true location shift is not equal to 0
95 percent confidence interval:
 -8.510 -0.689
sample estimates:
difference in location 
               -4.4715

In my example above, it gave an estimated location-shift of $-4.4715$ with a 95% interval for the shift $(-8.510, -0.689)$

The estimate of the probability that an X exceeds a Y is $U/n_1n_2 = 0.297$. [I don't know a suitable nonparametric way of generating an interval for that when it's different from $\frac{_1}{^2}$ though.]

As for the difficulty that you don't have the original $U$'s I don't know what can be done, but I'd suggest asking the original authors for the data... or at least the $U$'s.