I am struggling to figure out how should I approach to decide the sample size for a pilot study. The pilot study will be funded so a proposal is required justifying the sample size. The study participants are patients from a city hospital which will be extended to multiple cities in the main study. Patients will be evaluated based on the present of an attribute (this is not a randomized control trial, however, based on a character, we will have two groups of patients). Most of the measurements obtained will be numeric. From a previous study with a broader goal has a response rate of 40%.

I read some literature and liked the idea of deciding the sample size based on feasibility a good literature. I do have a confusion, though. When I calculate the sample size based on the proportion of nonresponse for a particular level of confidence, what does the margin of error mean here? Is is the margin of error for the estimates from my study or, the margin of error for the nonresponse rate? Any help is greatly appreciated! Thank you.

  • $\begingroup$ Since time and money are not constraints, enroll everybody everywhere all the time. Follow nonresponse up with personal visits and arm-twisting to get data from everyone. You might as well do this worldwide, since you have unlimited resources. If these recommendations (which are perfectly logical consequences of your assertions) strike you as ridiculous, then you need to back up and consider (1) the resources you might actually get and (2) what the objectives of this study are. We can't tell you anything useful until you disclose at least that much information. $\endgroup$
    – whuber
    Dec 21 '16 at 21:51
  • $\begingroup$ @whuber Did some editing... I hope you are a bit calmer now :) $\endgroup$ Dec 21 '16 at 22:09
  • 2
    $\begingroup$ What is the purpose of your pilot study? What do you hope to get at the end of this... what question are you trying to answer? Next, what are you going to measure in order to answer the question? Once you have a measure, how are you going to analyze your data and with what degree of precision and power do you need from your analysis to answer the question? If you know this... can you just work backwards to determine sample size? $\endgroup$
    – Matt S
    Dec 21 '16 at 22:18
  • $\begingroup$ @MattS Thank you, Matt, for your suggestion. I will look into this option to see if I can make it work. $\endgroup$ Dec 21 '16 at 22:38
  • 1
    $\begingroup$ Often pilot studies are used to determine the required sample size for the size next study. Sample size is often related to variance. So since the variance is often not known (approximately) the estimate from the pilot study is used. Of course to two studies should be close to identical. $\endgroup$ Dec 21 '16 at 23:08

The ultimate goal is to see, out of all these patients those who have a treadmill, for example, in the house are more physically active. We can have the average physical activity of these patients per week. We can also find the correlation of physical activity with well-being and test if patients with a treadmill have a higher correlation between physical activity and personal well-being.

I think in your case you would want to start with a simple cohort study as opposed to a randomized control trial (RCT). An RCT would be ideal (but more costly) for your eventual pivotal study. The goal of this cohort study would be to ask: does exposure to x (having a treadmill in the house) associate with outcome Y (increased physical activity)?

Such a study would recruit a group of people with a treadmill in the house and a group of people without a treadmill (the unexposed group) and follow them for a set period of time and note differences in the incidence of physical activity (PA) between the groups at the end of this time.

The groups should be matched in terms of other variables such as economic status and other health status so that the variable being assessed, the independent variable (in this case, owning a treadmill) can be isolated as the cause of the dependent variable (in this case, PA).

your statistical method would be based on your measure(s) of PA. One thing to consider is how many PA measures can you obtain from each cohort, and how often will you take these measures (daily, weekly, monthly).

From my perspective, focusing on a large group of PA measures and identifying their variance within the two groups and between the two groups would be critically important for this pilot study. You may find some measures vary a lot and would be terrible as metrics to design a future pivotal study, whereas others are less variable and would require a much smaller sample size in a future pivotal study to obtain an appropriate Power.

If you do perform any statistical significance testing using this pilot study data, take it with a grain of salt. The family-wise error rate will come into play. If multiple comparisons are done or multiple hypotheses are tested, the chance of a rare event increases, and therefore, the likelihood of incorrectly rejecting a null hypothesis (i.e., making a Type I error) increases.

Here there is an explanation and advice given on sample size estimations for cohort studies. Remember to consider any sample size will need to be adjusted to account for the expected drop-outs from your study.

Lastly, you may be able to perform a less costly Case-Control Study to determine if people that have higher levels of PA tend to own treadmills. It may be difficult to calculate your measures of PA on people based on a questionnaire or survey, but the study would be simpler to perform, because you would not need to find and follow people over time. However, it would be less valuable to help you plan for a pivotal study.

Hope this helps.

  • $\begingroup$ @jeffaltogether Thank you for your elaborate response! However, it will be a cross-sectional study. Mail surveys will be sent out to the participants and 40% of the participants are expected to have a treadmill in the house. $\endgroup$ Jan 2 '17 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.