When examining the difference between two proportions I typically use Cohen's h (i.e, the difference between two arcsin-transformed proportions) for the effect size.
Does anyone know how I could calculate 95% confidence intervals for Cohen's h?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
You can use this function (as I also implemented here):
cohens_h = function( prop_1, prop_2, n1, n2, ci = 0.95 ){
x1 = asin(sign(prop_1) * sqrt(abs(prop_1)))
x2 = asin(sign(prop_2) * sqrt(abs(prop_2)))
es = x1 - x2
se = sqrt(0.25 * (1 / n1 + 1 / n2 ))
ci_diff = qnorm(1 - (1-ci) / 2) * se
return( c( h = es*2, h_low = (es-ci_diff)*2, h_upp = (es+ci_diff)*2 ) )
}
E.g.:
acc1 = .70
acc2 = .90
N1 = 200
N2 = 300
cohens_h(acc1, acc2, N1, N2)
Returns:
h h_low h_upp
-0.5157784 -0.6946978 -0.3368590
-
$\begingroup$ Where is that formula coming from? $\endgroup$ Nov 30, 2021 at 2:04
-
$\begingroup$ Sorry, can't remember the original source. But via random checks it can be seen that the CI will correctly exclude/include zero corresponding to the proportion test's p value being above or below the relative alpha level. $\endgroup$– gasparDec 1, 2021 at 1:19
$\begingroup$
$\endgroup$
You can calculate 95% confidence intervals of arcsine difference, using the function "metabin" on the package "meta" in R. Cohen's h is the double of arcsine difference. You can calculate 95% confidence intervals of Cohen's h, the double of the lower and upper limit of confidence intervals of arcsine difference.
