5
$\begingroup$

I know that $r$ is itself a measure of the effect size, but I would like to know if using Spearman's rank test I can argue that the relation between X and Y is significant with $r = 0.33$ and that the effect is medium, as I do with Pearson test.

$\endgroup$
3
  • $\begingroup$ @clare Interpretation of ESs generally have no mathematical/statistical justification. $\endgroup$
    – chl
    Dec 18, 2010 at 18:01
  • $\begingroup$ @chl It is possible your remark could be misinterpreted. I think you are saying that one should seek non-mathematical justification of the interpretation of a particular ES, and not that you are saying interpreting an ES is altogether unjustified! $\endgroup$
    – whuber
    Dec 18, 2010 at 21:30
  • $\begingroup$ @whuber Indeed, I originally meant that interpreting ESs is highly task or domain-dependent (with some general rules of thumb for e.g., Cohen's d or Fleiss' Kappa), and that a good literature review is generally welcome. $\endgroup$
    – chl
    Dec 18, 2010 at 21:52

2 Answers 2

6
$\begingroup$

I see no obvious reason not to do so. As far as I know, we usually make a distinction between two kind of effect size (ES) measures for qualifying the strength of an observed association: ES based on $d$ (difference of means) and ES based on $r$ (correlation). The latter includes Pearson's $r$, but also Spearman's $\rho$, Kendall's $\tau$, or the multiple correlation coefficient.

As for their interpretation, I think it mainly depends on the field you are working in: A correlation of .20 would certainly not be interpreted in the same way in psychological vs. software engineering studies. Don't forget that Cohen's three-way classification--small, medium, large--was based on behavioral data, as discussed in Kraemer et al. (2003), p. 1526. In their Table 1, they made no distinction about the different types of ES measures belonging to the $r$ family. There have by no way an absolute meaning and should be interpreted with reference to established results or literature review.

I would like to add some other references that provide useful reviews of common ES measures and their interpretation.

References

  1. Helena C. Kraemer, George A. Morgan, Nancy L. Leech, Jeffrey A. Gliner, Jerry J. Vaske, and Robert J. Harmon (2003). Measures of Clinical Significance. J Am Acad Child Adolesc Psychiatry, 42(12), 1524-1529.
  2. Christopher J. Ferguson (2009). An Effect Size Primer: A Guide for Clinicians and Researchers. Professional Psychology: Research and Practice, 40(5), 532-538.
  3. Edward F. Fern and Kent B. Monroe (1996). Effect-Size Estimates: Issues and Problems in Interpretation. Journal of Consumer Research, 23, 89-105.
  4. Daniel J. Denis (2003). Alternatives to Null Hypothesis Significance Testing. Theory and Science, 4(1).
  5. Paul D. Ellis (2010). The Essential Guide to Effect Sizes. Cambridge University Press. -- just browsed the TOC
$\endgroup$
5
  • $\begingroup$ I only have a german text-book as a reference for it, but the authors state that a) Spearman's $r_{S}$ systematically overestimates true $\rho$ for normally-distributed data, and that b) $r_{S}$ is difficult to interpret when not near 0, 1 or -1. (Büning H & Trenkler G. 1994. Nichtparametrische statistische Methoden. 2. Aufl. Berlin: de Gruyter. With regards to a), they also state that typically, Kendall's $|\tau| <$ Spearman's $|r_{S}|$. $\endgroup$
    – caracal
    Dec 21, 2010 at 19:15
  • $\begingroup$ I edited too slowly... With regards to a), B&T cite Walter (1963) and also state that $r_{S}$ is only nonparametric under independency. Walter E. 1963. Rangkorrelation und Quadrantenkorrelation. Züchter Sonderhefte 6: Die Frühdiagnose in der Züchtung und Züchtungsforschung II, 7-11. $\endgroup$
    – caracal
    Dec 21, 2010 at 19:24
  • $\begingroup$ @caracal I guess Siegel's book is also a good reference. I also know that Kendall's $\tau$ has better statistical properties (but this has already been discussed in previous Q&A). But it seems to me that the question has to do with the interpretation of an ES, and although we're not informed of the sample size, it is likely to be small. Anyway, your proposed transformation should work fine and there exist transformation for converting an $r$-based measure into a Cohen's $d$ one. But as you didn't address directly the question, I wasn't sure of where you want to go. Now, I've +1 your response. $\endgroup$
    – chl
    Dec 21, 2010 at 22:32
  • $\begingroup$ Thanks! Actually, I deliberately tried to answer only the first part of the question (significance test of 0.33), since I wasn't sure about using $\rho$ as an ES measure. $\endgroup$
    – caracal
    Dec 21, 2010 at 22:48
  • $\begingroup$ @caracal Ok, I mostly focused on the other point. The references I gave are mainly opposing the Hypothesis Testing approach vs. interpretation of ES measures. I still prefer giving confidence intervals for $\hat r$ or $\hat\rho$, but anyway I hope the asker would find good directions for further investigations. $\endgroup$
    – chl
    Dec 21, 2010 at 22:55
3
$\begingroup$

With increasing sample size $n$, $r_{z} = \sqrt{n-1} r_{S}$ is asymptotically $N(0, 1)$ distributed (standard normal distribution). In R

rSz   <- sqrt(n-1) * rS
(pVal <- 1-pnorm(rSz))   # one-sided p-value, test for positive rank correlation
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.