Say I want to make a specific test. What is the correct terminology to describe the issue of having enough data to properly do it. Consider for instance:

"Of course the authors also needed to consider the issue of including enough data in order for their tests to be/have -INSERT WORD-".

One could of course say have the correct power/size, but this is specific issues so I guess I am looking for a kind of umbrella term meaning: without all the types of problems one encounters when the sample is too small.

  • 1
    $\begingroup$ Reliable perhaps? $\endgroup$ – JonB Sep 11 '15 at 10:07
  • $\begingroup$ @JonasBerge I actually like this, albeit I was expecting a statistical term. $\endgroup$ – Henrik Sep 11 '15 at 11:55
  • $\begingroup$ I don't think there is one that can always be used. $\endgroup$ – JonB Sep 11 '15 at 11:56
  • $\begingroup$ 'Reliable' is a test-retest concept, which is perhaps not what you are looking for. As per my comment to @tim, the relevant concept is indeed 'power' via 'estimation precision'. $\endgroup$ – conjugateprior Sep 11 '15 at 12:05

But the term is power

Power analysis can be used to calculate the minimum sample size required so that one can be reasonably likely to detect an effect of a given size. Power analysis can also be used to calculate the minimum effect size that is likely to be detected in a study using a given sample size.

(Source https://en.wikipedia.org/wiki/Statistical_power)

| cite | improve this answer | |
  • $\begingroup$ Well, what about a situation when the sample size is large enough to detect a difference, say an odds ratio of 1.8, but the confidence intervals may be very wide at 1.01-3.9. The power, as formally defined (might) be adequate but the sample size in this case is still too small to get a reliable estimate of the effect of the independent variable of interest. $\endgroup$ – JonB Sep 11 '15 at 10:28
  • $\begingroup$ @JonasBerge Then the test is able to correctly reject null but the confidence intervals are wide... Test works properly in here. $\endgroup$ – Tim Sep 11 '15 at 10:53
  • $\begingroup$ Yes of course, but it is a situation in which the sample size might be too small to provide a reliable estimate of the effect of the independent variable of interest. So, not an issue of power but still a too small sample size. The issue might be a new and expensive drug, and we then want to determine the size of the effect, not just that it is better than the existing preferred treatment. The sample size is too small to give a reliable estimate, but it does not lack the power to detect a difference. So problem with a small sample size isn't equivalent to lack of power? $\endgroup$ – JonB Sep 11 '15 at 11:12
  • 1
    $\begingroup$ I think @tim is correct that power is the right concept. Consider that, in addition to the model specification, power is a function of desired $\alpha$, effect size, and sample size. While it's traditional to fix model, $\alpha=0.05$ (or somesuch), and expected effect size, then compute a required sample size, the concept can be used other ways. Specifically, desired $\alpha$ is tightly connected to confidence interval width. So requiring a smaller $\alpha$ in the previous power computation indirectly demands a more precise i.e. narrower, intervals, which is what you want. $\endgroup$ – conjugateprior Sep 11 '15 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.