Sample size calculation without data

Question

I am writing a proposal for a research project and have been asked to calculate the sample size and power for the data I will use. The problem is that this will be an analysis of two repeated surveys of a sample of a country's population (panel panel) that I don't yet have access to (1500 respondents with complete data). What would be the best technical way of explaining this in my proposal rather than just saying "I don't have the data yet to do this". I am aware that the group I am submitting this proposal to are quite shrewd so I would like to state the theoretical reasons that I need the data as clearly and correctly as possible.

Answer 1

Why would you do a sample size calculation if you already know how many you're going to sample? Or will you need a subset of the 1,500?

There are 3 variations on the sample size calculation:

• Sample size calculation: power set, known effect, find $n$

• Power calculation: known $n$, known effect, find power

• Minimally detectable effect: power set, known $n$, find effect.

Each of these historically were used to justify going about research and have become a bit of a tradition rather than something which is taken seriously, unfortunately. Largely, this is because the guesswork that's involved, you have to make some assumptions based on the population (finding the $n$), or the literature (finding the effect size). Even the NIH is somewhat more drawn toward the "interestingness" of the research or the researcher more than the feasibility of the study, in my statistical perspective. People don't think of power as a continuous variable, though, just like people always seek $p < 0.05$ they also want power $> 80\%$.

When a study is going to be done one way or another, a power calculation is moot.

It sounds like you should present a minimally detectable effect. If you are sampling, say, 100 people and you want to set power to a "good" level and say, "oh we can reject a null hypothesis with probability (our good level) if the effect is this big or bigger." this is especially useful when there is no good literature out on the thing you're trying to measure.