1
$\begingroup$

We are trying to do a randomized controlled trial on patients. For a certain diseases patients, we want to know whether or not intervention improves their quality of life or not. We're dividing the patients into two groups: one receive the traditional method, other receives the intervention where their vitals and labs are checked with responses to some questionnaires about their quality of life is collected in 2 weeks, 4 weeks, 3 months, and 6 months. Whereas, for the traditional method, we're only going to follow up with patient in 1 month and 6 months. The lab and vitals for the traditional methods are only taken at 6 months.

For this, we'd like to know the power and sample size of the study. I would like to either know how to calculate it, or would like to know what I must know before I ask statistician's help. I read Cohen's article A Power Primer which help me make some sense, but I'm still somewhat confused about effect size

$\endgroup$
1
$\begingroup$

First, if you have the funds, I strongly suggest collecting pre-intervention data for the traditional method group. This will allow a stronger (more valid) study. Your current design suffers from a threat to ecological validity such that you lose ability to make causal claims. Also, again if you can afford it, add a control group.

In making the calculation you want, you need one of the numbers. Normally, you have a minimum detectable effect size that you set before hand, along with your acceptable $\alpha$ type-1 error rate and $1-\beta$ desired power. Having these values gives you the required $n$ sample size Such an analysis a a priori. On the other hand, if you have access to a restricted sample size $n$, acceptable $\alpha$ type-1 error rate, and $1-\beta$ desired power, you would learn your minimum detectable effect size, called sensitivity analysis.

I use G*Power, a free software, to do such calculations.

$\endgroup$
  • $\begingroup$ thank you. Effect size is what's causing me confusion. We don't have fund for the pre-intervention study, so was hoping to find any statistical method to find an estimate of the sample size. I'll download G*power $\endgroup$ – user1828605 Dec 11 '17 at 21:05
  • $\begingroup$ That was difficult for me, too. Think of it like a microscope. To see something smaller, you need stronger magnification. Likewise, to find a small effect size, you need either: a bigger sample, a bigger power, or a higher acceptable α. $\endgroup$ – Jay Schyler Raadt Dec 11 '17 at 21:38
2
$\begingroup$

Elaborating a bit on Jay's excellent answer and assuming that your experiment's outcome of interest is a proportion:

In order to determine a sample Size for a 2-Proportion $Z\text{-test}$ , you need to specify:

  1. the desired $α$ level
    • i.e., your willingness to commit a Type I error
  2. the desired $β$ level
    • i.e., your willingness to commit a Type II error
    • (this equivalent to desired power)
  3. a meaningful difference from the value of the parameter that is specified in the Null
    • i.e., you need to specify an "Alternative" hypothesis value

This is because rearranging the 2-Proportion $Z\text{-test}$ formula (for equal Treatment and Control group sizes) gives us the minimum sample size needed $\, \tilde{n}$:

$$\tilde{n} = (p_1(1-p_1)+p_2(1-p_2))\left(\dfrac{Z_{power}+Z_{1-\alpha}}{p_1-p_2}\right)^2$$

where

  • $p_1$ is your current proportion that has a high "quality of life"
  • $p_2$ is your current proportion that has a high "quality of life" + minimum effect size
    • this is the prespecified minimum proportional that you choose to be able to detect
  • $ \tilde{n}$ is the minimum total sample size needed
    • i.e., it is the sum of both the Treatment and the Control groups


w.r.t. not doing this calculation manually, one superb open-source solution is to use R-language's pwr library.

The specific function in this case is the pwr.2p.test

  • For any of pwr's functions, you enter three of the four quantities (effect size, sample size, significance level, power) and the fourth is calculated.
$\endgroup$
  • 1
    $\begingroup$ Excellent answer. Thank you. This is very helpful. $\endgroup$ – user1828605 Dec 13 '17 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.