1
$\begingroup$

I'm looking to do a pre-posttest RCT comparing difference of means through a t-test, however I'm completely unsure how to calculate a required sample size using power, confidence level, and standardized effect size.

From my understanding I need to use this formula: enter image description here

Then using this d value I can use a table to find the appropriate sample size. However, my main issue is determining the study to derive the X1 and X2 means in the formula from. Firstly, I'm assuming the test used to measure needs to match up (I.E. The study that I'm deriving the pre and post test means from need to use the same test as my trial, such as a VAS measure for pain) and obviously they both need to look at the same condition (Osteoarthritis in my case). However, does the study that I'm using the means from need to be completed on the same population and does the intervention need to be be the same dosage and duration or can a very similar dosage and duration be considered adequate?

Also, does the fact that I'm using an active control rather than a placebo have any influence on my calculation? How would modifying my trial to become a crossover design influence my sample size?

Thanks in advance, appreciate any support you guys can give me.

$\endgroup$
  • 1
    $\begingroup$ I think some clarification may be needed: ¿is this both a treatment/control group design and a pre-/post-test design? More specifically, I am wondering if the $\bar{x}$ for each group is is the average of the difference scores between the pre- and post-tests. $\endgroup$ – Gregg H Mar 23 '18 at 2:03
  • $\begingroup$ Hi Gregg, it's both a pre-/post test design with a treatment and an active control group. Yes, the x̅ for each group should be the average of the difference scores between pre- and post-tests groups. $\endgroup$ – mattb Mar 23 '18 at 2:56
  • $\begingroup$ One more follow-up: ¿do any of the articles report the standard deviation for the change from pre- to post-test? (even if it is from a report that doesn't include an experimental design) $\endgroup$ – Gregg H Mar 23 '18 at 3:48
  • $\begingroup$ Yes, there are some articles that report standard deviation. $\endgroup$ – mattb Mar 23 '18 at 5:05
2
$\begingroup$

Let’s say we can locate the following values from the literature:
    •  an estimate of the average VAS measure,
    •  an estimate of the standard deviation of VAS,
    •  an estimate for the pre- to post-test change in VAS, and
    •  an estimate for the standard deviation of the differences.
We will need to locate reported data on comparable populations, but the idea of using effect size estimates is just that...and preliminary estimate.  Thus, getting a reasonably comparable population is sufficient.  Additionally, because we are hoping to compare the change in VAS scores, any reasonable intervention that has a measurable difference is a reasonable stand in.  (I.e., to assess if this new intervention is at least as effect as [blank], we would need a sample size of ___.)

Again, as the focus is on comparing the change in VAS scores for an intervention vs. no-intervention, we can assume that no-intervention results in zero change.  Thus, $\bar{x}_2=0$.  $\bar{x}_1$ will be the estimate from the literature (3rd bullet above).

As we are planning a future study, we might as well plan a “perfect” future study, so we will assume $n_1 = n_2 = n$.  With this assumption, the pooled standard deviation estimate becomes $$s_\text{pool} = \sqrt{\frac{s_1^2 + s_2^2}{2}}$$ We will let $s_1$ be the estimated standard deviation of differences (from 4th bullet), and we will set $s_2^2$ equal to twice the general VAS standard deviation (from 2nd bullet).  (This is a quick-and-dirty estimate for the spread for difference scores assuming no difference...and we have to double the variance, not the standard deviation.)  With all of this, we should have enough to calculate the estimated target effect size $d$.

Lastly, if you are comparing an intervention to a different intervention, then you can argue that the effect size will be some portion of this target $d$, say 60%.  You can then use this rescaled $d$ in you power and sample size estimates.

I hope this is useful; happy to elaborate as need be.

$\endgroup$
  • $\begingroup$ Thanks! However,the main issue right now is determining which paper to derive my effect size d from. So from my understanding when calculating the d from data derived from previous research the only necessary requirement for choosing the paper is that it uses the following: the VAS scale and a similar population? Is this correct? I've found a similar study with similar invention, baseline VAS score and post treatment scores along with s. HOWEVER, it studied a different pathology (It studied a different disease and pain type other than what I'm interested in), is that still fine? (1/2) $\endgroup$ – mattb Mar 24 '18 at 9:02
  • $\begingroup$ Also, would the estimate of the average VAS measure just be the baseline or pre-treatment VAS score? (2/2) $\endgroup$ – mattb Mar 24 '18 at 9:04
  • 1
    $\begingroup$ (1) If there are no other published reports specifically addressing your population and pathology, then using estimates from what has been published—even if not exactly the same—for your estimation process is considered a reasonable approach. (2) I'm not sure I follow the query, as I would often call the pre-treatment measure the baseline measure. If you have multiple baselines measures, the overall average would probably suffice. $\endgroup$ – Gregg H Mar 24 '18 at 13:08
  • $\begingroup$ Thanks again, two more quick questions (Sorry for being persistent). (1/2) I found a metaanalysis which looked at pain reduction of opioids for osteoarthritis and there was specifically a section on codeine however when checking the papers individually they all used different pain measures other than VAS. Is it possible to somehow calculate a sample size that I can use in my paper from this? Link: imgur.com/M8CZeTu $\endgroup$ – mattb Mar 25 '18 at 9:18
  • 1
    $\begingroup$ Q1: yes, you can argue that the effect size calculated form one study should be a comparable goal for another study (not exactly using the same means & SDs, but the resulting $d$ can be used). Q2: Use this formula, $SE = \frac{SD}{\sqrt{n}}$ or $SE\sqrt{n}=SD$ to approximate the $SD$. $\endgroup$ – Gregg H Mar 25 '18 at 13:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.