0
$\begingroup$

I am planning a study and for that I need to use power analysis to estimated the sample size. Now, I do not have any estimates for the desired effect and standard deviation required to do the power calculation. I also cannot do a pilot study. Given these constraints, how do I estimate the sample size for my study?

$\endgroup$
2
$\begingroup$

I don't know, how hungry I will be, nor, how many people will join the meal. How much food should I buy?

You cannot do any reasonable computation without suitable data. You could probably try to guess reasonably. Or you need to look for similar investigations that have been done by others or you have yourself guided by what $n$ is reasonably achievable. Or maybe a descriptive analysis is in order?

Also look into How to choose the sample size for a pilot study for making a power analysis?

$\endgroup$
  • $\begingroup$ I tried to calculate sample size for a paired t-test study and I get the following results. I used the means and std's from an actual and similar study. The sample size I get is 16. I do not understand these results. Can someone please explain? Input: Tail(s) = Two Effect size dz = 0.7596495 α err prob = 0.05 Power (1-β err prob) = 0.80 Total sample size = 16 $\endgroup$ – SZA Feb 7 '17 at 20:05
  • $\begingroup$ Two-tailed tests are the standard. If you don't have very good reasons, you will want to do a two tailed test. $\alpha$ = .05 means that you consider your result significant, if p is below .05, the power is usually set to 80 or 90% giving the chance to find an effect as big as given with this $n$ is 80% or 90%. If you can gather more than 16, increasing the power over 80% isn't strictly bad. The effect size to be detected has been computed from the difference of the means and the standard deviations. Go with something slightly over $n = 16$, maybe twenty, if you can afford that. $\endgroup$ – Bernhard Feb 7 '17 at 21:13
2
$\begingroup$

While Bernhard is right that having some more info or a pilot would be ideal, a pragmatic approach in a situation where neither is available would be

  1. Fix your question
  2. Fix your statistical analysis
  3. Decide on the minimal effect size that is of SCIENTIFIC significance, i.e. that you want to be able to pick up if it is there, and use this as effect size in the power analysis
  4. Get an estimate about variability from the literature / colleagues (very unlikely that there is no precedence for a measurement method)
$\endgroup$
  • $\begingroup$ In some cases it may be possible to reduce steps 3 and 4 to a single number (e.g. "population signal to noise ratio", Cohen's $d$). $\endgroup$ – GeoMatt22 Feb 6 '17 at 20:25
  • $\begingroup$ Right, assuming there are studies that have looked at similar effects. $\endgroup$ – Florian Hartig Feb 6 '17 at 20:39
3
$\begingroup$

The short answer is that you can't. Like other answers mentioned you need to assume a fixed effect size, fixed significance level and a desired statistical power in order to estimate the needed sample size.

Now if you don't know the mean difference and standard deviation - you can guess them based on some information.

And if you cannot reasonably do that then the next best thing is to calculate required sample sizes for a whole range of possible mean differences and standard deviations. This would allow you to at least imply on a range of possibilities. And you would be able to answer questions such as "what kind of effects can I reasonably (i.e. with power = 80%) expect to detect with a sample size of n".

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.