10
$\begingroup$

Quick question: I've seen Cohen's d calculated two different ways for a dependent samples t-test (e.g., within-samples design testing the efficacy of a medication with pre/post timepoints).

  1. Using the standard deviation of the change score in the denominator of the equation for Cohen's d.
  2. Using the standard deviation of the pretest score in the denominator of the equation for Cohen's d.

I've found very little literature actually delineating which to use and/or when to use either option.

Any quick thoughts?

$\endgroup$
6
+25
$\begingroup$

Geoff Cumming has a few comments on the matter (taken from Cumming, 2013):

In many cases, however, the best choice of standardizer is not the SD needed to conduct inference on the effect in question. Consider, for example, the paired design, such as a simple pre–post experiment in which a single group of participants provide both pretest and posttest data. The most appropriate standardizer is virtually always (Cumming, 2012, pp. 290–294; Cumming & Finch, 2001, pp. 568–570) an estimate of the SD in the pretest population, perhaps $s_1$, the pretest SD in our data. By contrast, inference about the difference requires $s_{diff}$, the SD of the paired differences—whether for a paired t test or to calculate a CI on the difference (Cumming & Finch, 2005). To the extent the pretest and posttest scores are correlated, $s_{diff}$ will be smaller than $s_1$, our experiment will be more sensitive, and a value of d calculated erroneously using $s_{diff}$ as standardizer will be too large.

The primary reason for choosing $s_{pre}$ as standardizer in the paired design is that the pretest population SD virtually always makes the best conceptual sense as a reference unit. Another important reason is to get d values that are likely to be comparable to d values given by other paired-design experiments possibly having different pretest–posttest correlations and by experiments with different designs, including the independent-groups design, all of which examine the same effect. The d values in all such cases are likely to be comparable because they use the same standardizer—the control or pretest SD. Such comparability is essential for meta-analysis, as well as for meaningful interpretation in context.

$\endgroup$
3
$\begingroup$

I found the formal answer in Frontiers in Psychology. If $t$ is the test statistic, and $N$ is the number observations, then:
$$ d = \frac{t}{\sqrt{N}} $$

$\endgroup$
  • $\begingroup$ Note that this will give you the standardized mean change where the mean change is standardized in terms of the the standard deviation of the change scores (what is denoted as 1. in the question). $\endgroup$ – Wolfgang Sep 5 '16 at 15:24
0
$\begingroup$

Here is a suggested R function that compute Hedges' g (the unbiased version of Cohen's d) along with its confidence interval for either between or within-subject design:

gethedgesg <-function( x1, x2, design = "between", coverage = 0.95) {
  # mandatory arguments are x1 and x2, both a vector of data

  require(psych) # for the functions SD and harmonic.mean.

  # store the columns in a dataframe: more convenient to handle one variable than two
  X <- data.frame(x1,x2)

  # get basic descriptive statistics
  ns  <- lengths(X)
  mns <- colMeans(X)
  sds <- SD(X)

  # get pairwise statistics
  ntilde <- harmonic.mean(ns)
  dmn    <- abs(mns[2]-mns[1])
  sdp    <- sqrt( (ns[1]-1) *sds[1]^2 + (ns[2]-1)*sds[2]^2) / sqrt(ns[1]+ns[2]-2)

  # compute biased Cohen's d (equation 1) 
  cohend <- dmn / sdp

  # compute unbiased Hedges' g (equations 2a and 3)
  eta     <- ns[1] + ns[2] - 2
  J       <- gamma(eta/2) / (sqrt(eta/2) * gamma((eta-1)/2) )
  hedgesg <-  cohend * J

  # compute noncentrality parameter (equation 5a or 5b depending on the design)
  lambda <- if(design == "between") {
    hedgesg * sqrt( ntilde/2)
  } else {
    r <- cor(X)[1,2]
    hedgesg * sqrt( ntilde/(2 * (1-r)) )
  }

  # confidence interval of the hedges g (equations 6 and 7)
  tlow <- qt(1/2 - coverage/2, df = eta, ncp = lambda )
  thig <- qt(1/2 + coverage/2, df = eta, ncp = lambda )

  dlow <- tlow / lambda * hedgesg 
  dhig <- thig / lambda * hedgesg 

  # all done! display the results
  cat("Hedges'g = ", hedgesg, "\n", coverage*100, "% CI = [", dlow, dhig, "]\n")

}

Here is how it could be used:

x1 <- c(53, 68, 66, 69, 83, 91)
x2 <- c(49, 60, 67, 75, 78, 89)

# using the defaults: between design and 95% coverage
gethedgesg(x1, x2)

# changing the defaults explicitely
gethedgesg(x1, x2, design = "within", coverage = 0.90 )

I hope it helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.