Select correct effect size (Cohen's d) for power analysis

Question

According to Blevins et al. (2015) the Posttraumatic Stress Disorder Checklist for Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (PCL-5) shows internal consistency alpha values of .94 and .95 in its 2 studies. The authors show that the test-retest reliability of the instrument was .82 (which seems to be considered as strong) indicating stable and consistent results over time. The convergent validity, the authors mention, showed high correlations with posttraumatic stress disorder (PTSD) measures (e.g., .85 with PCL, .85 with PDS, and .84 with DAPS) indicating that the PCL-5 seems to be measuring the same construct as other established PTSD measures. The instrument, the authors say, showed moderate correlations with related constructs (e.g., with depression r = .60) and lower correlation with unrelated constructs (e.g., r = .31 with mania) all of which seems to show a good discriminant validity.

My aim is to use the PCL-5 (Blevins et al., 2015) in a study in order to measure the level of PTSD symptoms in a sample of people working in armed conflict context.

However, before doing that, I need to carry out a power analysis to calculate an a priori minimum sample size I would need in order to achieve a reliable result. For that I need the following:

• Significance level (α = .05)
• Power (1-β = .80)
• Cohen's d (unknown yet)

My questions:

• Based on the description given above, can I decide on a d effect size myself?
• If yes, what value and how can I justify it (reference(s))?
• If no, what is/are the option/s and/or alternative/s?

Answer 1

No, this information, by itself, does not let you pick an effect size. You say you want to "carry out a study" using the measure. But the type of effect size to use will depend on the type of study. It might be Cohen's d, but it might be something else.

And the size of the effect size depends on how big an effect you want to be able to detect. This choice is not really a statistical one but a substantive one. You can look at the effect sizes that other, similar studies, have found. If that's not possible, then you have to pick one that is big enough to make a difference but small enough to be believable.

Suppose you were studying the effect of diet on weight loss in human adults. If you chose an effect size of 1 pound (or .5 kg) in a month, no one would care much. On the other hand, if you claimed one of 30 pounds in a month, no one would believe you.

You are the expert on PTSD. What effect size do you think is reasonable for whatever it is you are going to do in your study?