4
$\begingroup$

I am trying to recreate sensitivity calculation in Python by applying the same methodology as used in G*Power

z-tests > Proportions: Difference Between Two Independent Proportions > Sensitivity: Compute the required effect size

enter image description here

But I am not able to find any information on the underlying formula used, even in the official documentation. The only thing I've found in the software itself is that it uses "Cohen's h" for calculating the effect size.

Would appreciate any help regarding the calculations. I want to do these in Python but it doesn't looks like it has a library for that. Statsmodels are only able to perform power analysis with Cohen's d.

$\endgroup$
2
  • $\begingroup$ Strong hint: the effect sizes $h$ shown in your two screen shots are in the ratio $\sqrt{100/1000},$ the inverse square roots of the sample sizes. $\endgroup$
    – whuber
    Nov 25, 2023 at 17:53
  • $\begingroup$ @whuber sorry, I can't make anything of it. I'm not too strong on statistics unfortunately $\endgroup$ Nov 25, 2023 at 17:58

1 Answer 1

1
$\begingroup$

Statsmodels does not have a dedicated function for this.

However, the z-test is based on generic normal test for the difference of the (transformed) means of two independent samples.

import statsmodels.stats.power as smp

smp.NormalIndPower().solve_power(effect_size=None, power=0.8, nobs1=1000, alpha=0.1, ratio=1, alternative='larger')
0.09495162823574568

smp.NormalIndPower().solve_power(effect_size=None, power=0.8, nobs1=100, alpha=0.1, ratio=1, alternative='larger')
0.30026052525013747

R package pwr has an equivalent function, e.g. for computing power p = pwr.2p.test(h=0.3, n=80, sig.level=0.05, alternative="greater")

Aside: the initial statsmodels power computation were written as equivalents for R package pwr and G*Power.
see for example section "Tests for two Proportions (Normal Approximation)" in https://jpktd.blogspot.com/2013/03/statistical-power-in-statsmodels.html
More recent additions to power and sample size functions in statsmodels were often based on the documentation of NCSS/PASS.

background: https://en.wikipedia.org/wiki/Cohen%27s_h

Also, there other methods for comparing two proportions that have better statistical properties, e.g. available as method argument in statsmodels.
https://www.statsmodels.org/dev/generated/statsmodels.stats.proportion.test_proportions_2indep.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.