1
$\begingroup$

I was calculating effect sizes for an analysis and noticed by chance that Pearson's $r$ calculated from Cohen's $d$ with a conversion formula is not the same as when I calculate the correlation given the raw data. Does anyone have an idea why it is like this? I've tried with random numbers several times and the two $r$'s are not even close, I've also tried to calculate manually and looked up which formulas are used by the packages (so this does not seem like a problem with the package), I can't make sense of this.

np.random.seed(10)    
df = pd.DataFrame(np.random.randint(0,100,size=(100, 2)), columns=list('AB'))
d = pg.compute_effsize(df['A'],df['B'], paired=False, eftype='cohen')
r_calc = pg.compute_effsize(df['A'],df['B'], paired=False, eftype='r')
r_conv = pg.convert_effsize(d,'cohen','r')

EDIT: The formulas I tried for $d$ are $d = \frac{\bar{x}_{1} - \bar{x}_{2}}{s}$.

With $s = \sqrt{\frac{(n_1 - 1)s^2_1 + (n_2 - 1)s^2_2}{n_1 + n_2 - 2}}$, or $s = \sqrt{\frac{(n_1 - 1)s^2_1 + (n_2 - 1)s^2_2}{n_1 + n_2}}$.

(pg.compute_effsize uses the first version with -2 in the denominator).

I get $d = 0,206012$ (or $0,207049$ for the second formula) and $r = 0,1029749$ when I convert $d$ with the formula used by pg.convert_effsize

$r = \frac{d}{\sqrt{d^2 + \frac{(n_x + n_y)^2 - 2(n_x+ n_y)}{n_x n_y}}}$

(using the alternative $d$, I get $r = 0,103488$).

When I calculate $r$ as the Pearson correlation, I get $0,157920$.

This was already asked here but not answered unfortunately: Why does Pearson's $r$ differ from the converted value of $r$ from Cohen's $d$

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ There are many different formulas for Cohen's d, depending on the circumstances and assumptions. Please, then, tell us what formula you are using and what formula pg.convert_effsize might be using. $\endgroup$
    – whuber ♦
    Sep 17, 2022 at 19:56
  • $\begingroup$ @whuber I edited the post accordingly and added a random seed to make the example reproducible $\endgroup$
    – RBG
    Sep 17, 2022 at 21:20

1 Answer 1

Reset to default
3
$\begingroup$

The comment by @mdewey is the answer:

The formula for $r$ from $d$ gives you the point bi-serial correlation not Pearson's correlation coefficient.

To derive $r$ from $d$, compute the point-biserial correlation $r_{\operatorname{pb}}$ instead of the Pearson correlation coefficient $r_{\operatorname{pearson}}$ and use the bias correction term $\sqrt{(N-2)/N}$.

As an aside, the pingouin documentation is lacking since it doesn't define $r$ clearly. Cofusingly, the $r$ in pingouin.compute_effsize is not the same as the $r$ in pingouin.convert_effsize.

import numpy as np
import pingouin as pg

nx = ny = 100

np.random.seed(1234)

x = np.random.randint(0, 100, nx)
y = np.random.randint(0, 100, ny)

Compute summary statistics necessary to do the calculation by hand.

# Sample means and sums of squares for the two groups separately
xbar, ybar = np.mean(x), np.mean(y)
SSx, SSy = np.sum((x - xbar)**2), np.sum((y - ybar)**2)
SSxy = np.sum((x - xbar) * (y - ybar))

# The overall sample mean and sum of squares assume
# that the two samples come from the same population
mean = (nx * xbar + ny * ybar) / (nx + ny)
SS = np.sum((x - mean)**2) + np.sum((y - mean**2)

bias_correction = np.sqrt((nx + ny - 2) / (nx + ny))

In general Pearson's correlation coefficient is not equal to the point-biserial correlation.

# Same as np.corrcoef(x, y)[0, 1]
r_pearson = SSxy / np.sqrt(SSx * SSy)
r_pearson
#> 0.07740094

r_pb = (xbar - ybar) / np.sqrt(SS / (nx + ny)) * np.sqrt(nx * ny) / (nx + ny)
r_pb
#> 0.06503982

The point-biserial correlation is the Pearson correlation between the two samples $x$ and $y$ concatenated into one and a binary indicator variable for group membership.

np.corrcoef(np.concatenate([y, x]), np.repeat([0, 1], [ny, nx]))
#> array([[1.        , 0.06503982],
#>       [0.06503982, 1.        ]])

Now that we know the difference between Pearson's correlation and point-biserial correlation, we are ready to convert Cohen's $d$ to $r_{\operatorname{pb}}$ and vice versa. Note: We reverse the bias correction applied to Cohen's $d$, so that the converted effect sizes are equal to the input effect sizes.

d = pg.compute_effsize(
    x, y,
    eftype="cohen"
)
d
#> 0.13035565
#> 0.07740094
r = pg.compute_effsize(
    x, y,
    eftype="r"
)
r

d = d / bias_correction

d_to_r = pg.convert_effsize(
    d,
    "cohen", "r"
)
r_to_d = pg.convert_effsize(
    r_pb,
    "r", "cohen"
)
r_pb, d_to_r
#> (0.06503982, 0.06503982)
d, r_to_d
#> (0.13035565, 0.13035565)
Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you so much for the detailed answer, now it makes sense! So when textbooks and papers say that Pearson's r can be used as an effect size, they always mean the point biserial? E.g. when I need an effect size of a t-test, I thought I could correlate the values for the two groups, but indeed that makes not much sense. So instead I correlate all values with the dummy variables for the groups? Then essentially the way pg.compute_effsize calculates r is wrong? $\endgroup$
    – RBG
    Sep 18, 2022 at 15:23
  • 1
    $\begingroup$ I wouldn't say that pg.compute_effsize is wrong but that it's documentation is not clear enough... Instead think about what you actually want to compute. For example: Effect sizes are differences in group means. You should be able to calculate an effect size even if the two samples have different size and are not paired. Can you do this with Pearson's correlation? See the formula for $r_{\operatorname{pearson}} = SS_{xy} / \sqrt{SS_x * SS_y}$ and in particular how we compute the sum of squares $SS_{xy}$. $\endgroup$
    – dipetkov
    Sep 18, 2022 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.