# Measures of minimal detectable effect size

Statistical power is defined as the probability of wrongly accepting the null hypothesis for a given sample size, p-value cutoff and effect size.

I am after accepted measures of the converse: the minimum effect size that can be reliably detected given sample size, p-value cutoff and the desired level of reliability of detection.

I'll assume that by "level of reliability of detection", you simply mean power.

If so, I don't think there is anything more accepted than "minimum detectable effect size". As in:

By simulation, we found that the minimum effect size (Cohen's $d$) between two groups of size $n=20$ detectable using an unpaired t-test assuming equal variances with $\alpha=0.05$ and power $\beta=0.80$ is $d=0.91$.

Below is code I'd use. I'd change the value for dd until I got the power I wanted. (If you want to be fancy, you can wrap this search around the code.)

nn <- 1e5
kk <- 20
dd <- 0.91
alpha <- 0.05

detect <- rep(FALSE,nn)
for ( ii in 1:nn ) {
xx <- rnorm(kk)
yy <- rnorm(kk,dd)
detect[ii] <- t.test(xx,yy)$p.value<alpha } sum(detect)/nn # change dd until this is close to the desired power  • This is basically what I have in mind except for Cohen's$d$. I had in mind studies where you compare the probabilities of a (usually rare) event in the two groups. Things like probabilies of ad clicks, deaths, loan defaults, ... So the question is how big a relative or absolute change in probability can you detect for a given$\alpha$,$\beta$and$n\$. Often you have good estimate of the baseline probabilty. Apr 7, 2016 at 7:29
• I'd happily adapt a simulation like this to any measure of effect size - (absolute) differences in probabilities or odds, true coefficient values, classification measures like sensitivity or specificity, whatever. Then again, I have never tried putting something like this into a paper submission or project proposal. "Normal" power calculations, where you prespecify the effect size and calculate the necessary sample size, are more common. Apr 7, 2016 at 7:32