3
$\begingroup$

When examining the difference between two proportions I typically use Cohen's h (i.e, the difference between two arcsin-transformed proportions) for the effect size.

Does anyone know how I could calculate 95% confidence intervals for Cohen's h?

$\endgroup$

2 Answers 2

2
$\begingroup$

You can use this function (as I also implemented here):

cohens_h = function( prop_1, prop_2, n1, n2, ci = 0.95 ){
    x1 = asin(sign(prop_1) * sqrt(abs(prop_1)))
    x2 = asin(sign(prop_2) * sqrt(abs(prop_2)))
    es = x1 - x2
    se = sqrt(0.25 * (1 / n1 + 1 / n2 ))
    ci_diff = qnorm(1 - (1-ci) / 2) * se
    return( c( h = es*2, h_low = (es-ci_diff)*2, h_upp = (es+ci_diff)*2 ) )
}

E.g.:

acc1 = .70
acc2 = .90
N1 = 200
N2 = 300
cohens_h(acc1, acc2, N1, N2)

Returns:

         h      h_low      h_upp 
-0.5157784 -0.6946978 -0.3368590 
$\endgroup$
2
  • $\begingroup$ Where is that formula coming from? $\endgroup$ Nov 30, 2021 at 2:04
  • $\begingroup$ Sorry, can't remember the original source. But via random checks it can be seen that the CI will correctly exclude/include zero corresponding to the proportion test's p value being above or below the relative alpha level. $\endgroup$
    – gaspar
    Dec 1, 2021 at 1:19
1
$\begingroup$

You can calculate 95% confidence intervals of arcsine difference, using the function "metabin" on the package "meta" in R. Cohen's h is the double of arcsine difference. You can calculate 95% confidence intervals of Cohen's h, the double of the lower and upper limit of confidence intervals of arcsine difference.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.