16
$\begingroup$

According to Fritz, Morris, and Richler (2011; see below), $r$ can be calculated as an effect size for the Mann-Whitney U-test using the formula $$ r = \frac{z}{\sqrt N} $$ This is convenient to me, as I report $r$ also on other occasions. I'd like to report the confidence interval for $r$ in addition to the effect size measure.

Here are my questions:

  • Can I calculate the confidence intervals for r as for Pearson's r, although it is used as an effect size measure for a nonparametric test?
  • What confidence intervals have to be reported for one-tailed vs. two-tailed testing?

Edit concerning the second question: "What confidence intervals have to be reported for one-tailed vs. two-tailed testing?"

I found some more information that IMHO may answer this question. "Whereas two-sided confidence limits form a confidence interval, their one-sided counterparts are referred to as lower or upper confidence bounds." (http://en.wikipedia.org/wiki/Confidence_interval). From this information I conclude that it is not the main issue whether the significance testing (e.g., $t$-test) was one- or two-tailed, but what information one is interested in with respect to the CI for the effect size. My conclusion (please correct me if you disagree):

  • two-sided CI $\rightarrow$ interested in upper and lower bounds (as a consequence, it is possible that a two-sided CI entails 0 although the one-tailed test of significance was p < .05, especially in case the value was close to .05.)
  • one-sided "CI" $\rightarrow$ only interested in upper or lower bound (due to theoretical reasoning); however, this is not necessarily the main question of interest after testing a directed hypothesis. A two-sided CI is perfectly appropriate if the focus is on the possible range of an effect size. Right?

See below for the text passage from Fritz, Morris, & Richler (2011) on effect sizes estimate for the Mann-Whitney test from the article I refer to above.

"Most of the effect size estimates we have described here assume that the data have a normal distribution. However, some data do not meet the requirements of parametric tests, for example, data on an ordinal but not interval scale. For such data, researchers usually turn to nonparametric statistical tests, such as the Mann–Whitney and the Wilcoxon tests. The significance of these tests is usually evaluated through the approximation of the distributions of the test statistics to the $z$ distribution when sample sizes are not too small, and statistical packages, such as SPSS, that run these tests report the appropriate $z$ value in addition to the values for $U$ or $T$; $z$ can also be calculated by hand (e.g., Siegel & Castellan, 1988). The $z$ value can be used to calculate an effect size, such as the $r$ proposed by Cohen (1988); Cohen’s guidelines for r are that a large effect is .5, a medium effect is .3, and a small effect is .1 (Coolican, 2009, p. 395). It is easy to calculate $r$, $r^2$, or $\eta^2$ from these $z$ values because $$ r = \frac{z}{\sqrt N} $$ and $$ r^2\quad{\rm or}\quad \eta^2 = \frac{z^2}{N} $$ These effect size estimates remain independent of sample size despite the presence of N in the formulas. This is because z is sensitive to sample size; dividing by a function of N removes the effect of sample size from the resultant effect size estimate." (p. 12)

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
9
$\begingroup$

One choice of effect size for the Mann-Whitney U test is the common language effect size. For the Mann-Whitney U, this is the proportion of sample pairs that supports a stated hypothesis.

A second choice is the rank correlation; because the rank correlation ranges from -1 to +1, it has properties that are similar to the Pearson r. In addition, by the simple difference formula, the rank correlation is the difference between the common language effect size and its complement, a fact that promotes interpretation. For example, if there are 100 sample pairs, and if 70 sample pairs support the hypothesis, then the common language effect size is 70%, and the rank correlation is r = .70 = .30 = .40. A clear discussion of the common language effect size and of four formulas to compute the rank correlation is given by Kerby in the journal Innovative Teaching: Kerby (2014) Innovative Teaching

By the way, though the paper does not mention it, I am fairly certain that Somers d and the rank correlation for the Mann-Whitney are equivalent.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Do you mean "For example, if there are 100 possible pairs"? The Mann-Whitney U-test is for unpaired data, so the phrasing is ambiguous--you may want to clarify for readers what the possible pairs are. $\endgroup$
    – gung - Reinstate Monica
    Apr 24, 2015 at 19:49
  • 1
    $\begingroup$ Thanks for the comment and the chance to clarify. I referred to sample pairs. If there are 10 observations in the experimental sample, and if there are 10 observations in the control sample, then there are 10 * 10 = 100 sample pairs. According to Robert Grissom, the effect size of the sample is an unbiased estimator of the population effect size. Thus, if the rank correlation is r = .40 for the sample, this is an unbiased estimator of the population effect size. $\endgroup$
    – DSK
    Apr 24, 2015 at 22:13
  • $\begingroup$ I suspected that was what you meant, @DSK. I think that explanation will help people. You may want to edit that into your answer. Welcome to CV. $\endgroup$
    – gung - Reinstate Monica
    Apr 24, 2015 at 22:18
8
$\begingroup$

Your link leads me to an opportunity to purchase the article.

More fundamental to the Wilcoxon-Mann-Whitney 2-sample test is the concordance probability, which is a pure measure of separation of the two groups. You can get confidence intervals for this $c$-index a variety of ways, e.g., in R the Hmisc package rcorr.cens function which does not assume equal variance in the two groups. You can also convert $c$ to Somers' $D_{xy}$ rank correlation through $D_{xy} = 2\times (c - \frac{1}{2})$.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for bringing this to my notice (link). I've now inserted the passage on the Mann-Whitney test in my question. $\endgroup$
    – grey
    Nov 18, 2014 at 13:20
  • $\begingroup$ Thank you very much for your answer. Do you possibly have a link at hand about how to interpret the c-index and Somers' D? I'd be especially interested in whether the latter can be interpreted comparable to r. I have two samples and in the second sample (larger N and normal distribution) I report r. I think it would make the comparison of the results easier if the measures used were similar - as far as possible of course. That's why I was interested in the formula mentioned by Fritz et al. (2011). So the CI for their r cannot be calculated as for Pearson's r I assume? Many thanks again! $\endgroup$
    – grey
    Nov 18, 2014 at 13:39
  • $\begingroup$ I wasn't able to read that paper but I think that basing an index on a $z$ statistic may not be optimal. You can use $D_{xy}$ for binary, ordinal, and continuous $Y$. I'm not sure what is the best tutorial for $D$ and $c$; perhaps others can chime in. $\endgroup$
    – Frank Harrell
    Nov 18, 2014 at 20:00
  • $\begingroup$ Many thanks for your response. I searched for some more information on how to interpret Somer's, but I haven't been too successful so far. Can Somer's d be understood similar to Pearson's correlation coefficient, e.g. does squaring it yield a coefficient of determination? I'd be very happy to find an effect size measure that can be interpreted similarly to r, if one exists. $\endgroup$
    – grey
    Nov 18, 2014 at 22:59
  • $\begingroup$ I found some more information on the formula r = Z/√(N): Rosenthal (1991) writes that "we can usefully estimate an effect size r from a p level alone as long as we know the size of the study (N). We convert the obtained p to its standard normal deviate equivalent using a table of Z values." $\endgroup$
    – grey
    Nov 19, 2014 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.