I want to estimate the sample size needed to compare binary incidence rate between two populations (based on binary separation to low/high risk groups).

The known (previous research) incidence rate in general population is very low, 0.1%. While in the data I have for the retrospective research it is around 10%, due to the way the data for the research was collected.

What anticipated incidence rates should I use for the sample size calculations? How does the known general population incidence rate come into play?

  • 1
    $\begingroup$ Dear @Xyand could you please be more specific (hypothesis, sampling procedure used etc.)? $\endgroup$ – Ladislav Naďo Dec 23 '19 at 22:53
  • 1
    $\begingroup$ Can you also please state what is the ultimate target of this analysis? Do we care for the accuracy of the logit coefficients or the overall incident rate in a new population? $\endgroup$ – usεr11852 Dec 25 '19 at 1:48

I suspect that what you have estimated from your retrospective data is "prevalence," not "incidence." This distinction is explained for example in this paper. "Incidence" is the rate at which a condition occurs, for example the fraction of a population that develops the condition per year. The fraction of people that currently has the condition, whenever it first occurred, is "prevalence." Although it might be possible to use retrospective data to examine incidence, if you simply collect retrospective data on a set of patients and determine the fraction of them that had the condition, you are examining prevalence not incidence.

What follows, however, is the same regardless of whether you are examining incidence or prevalence.

The paper cited above points out that:

... all epidemiological studies are (or should be) based on a particular population (the ‘source population’) followed over a particular period of time (the ‘risk period’).

Your estimate of sample size thus needs to based on the "source population" from which you are sampling. In retrospective clinical data analysis you are "sampling" (typically, taking all cases) from the population that happens to have shown up for clinical care and thus is included in the data set. If that group of patients is your source population then you should use the characteristics of those patients as your guide to study design.*

The problem you face, as noted in a comment on your question, is extrapolation to the general population. You cite a 100-fold difference in "incidence" between the population from which you are sampling and the general population. That would seem to be a potentially serious problem.**

So if you wish to make any statements about the general population rather than just the "source population" that underlies your retrospective data, you must take the difference between the populations into account. As the above paper notes on page 395:

... some prevalence studies may involve sampling on exposure status, just as some incidence studies may involve such sampling. For example, in a study of a group of factory workers, asthma prevalence may be measured in all exposed workers and a sample of non-exposed workers. This sampling scheme does not change the basic study type, rather it redefines the population that is being studied (from the entire group of workers in the factory to the newly defined subgroup).

You might think about your situation as over-sampling the disease cases, similar to what's described in the preceding quote. But for the results to be interpretable in terms of the general population, you would have to document that both the disease cases and the non-disease cases in your "source population" are representative of what's in the general population. Given the apparently large difference in prevalence/incidence that you note and my experience with analysis of retrospective clinical data, my guess is that the characteristics of the non-disease cases in your data will a good deal different from the general population and that you will have to take that difference into account in your study.

*In single-institution retrospective analysis, trying to get a larger sample size generally means going back farther in time for more cases. Among other things, you then need to see whether there have been changes over time in incidence/prevalence or in the characteristics/risk factors of the retrospective-patient "source population."

**Some of the magnitude of this discrepancy might be due to a difference between incidence and prevalence, for example if this is a long-term condition and the value of 0.1% for the general population that you cite is truly an incidence rate (say per 100,000 people per year) and the 10% value you have estimated from your retrospective data is prevalence. Nevertheless, there would still seem to be some difference between your "source population" and the general population.

| cite | improve this answer | |
  • $\begingroup$ Can you please be a bit more specific on your suggestions? For me this reads mostly like an extended comment. Are there any relevant references on "sampling/survey misspecification" that you might be aware of? (Disclaimer: I really like your answers and I learn a lot out of them.) $\endgroup$ – usεr11852 Dec 28 '19 at 9:06
  • $\begingroup$ @usεr11852saysReinstateMonic thanks for the suggestion and the support. My response was mostly based on my experience/frustration with working on retrospective clinical databases, which has occupied much of my attention for several years. Clinical databases (in the US at least, where there is no common medical-record system) typically represent people who have presented to a specific clinical practice or hospital for treatment. They thus might not well represent the broader population, in many critical respects. I will look for a more formal reference. $\endgroup$ – EdM Dec 28 '19 at 13:47
  • $\begingroup$ @usεr11852saysReinstateMonic I added a pertinent reference that also helped improve the organization of the answer. Also saw I had missed that the retrospective rate cited by the OP was probably a prevalence rather than an incidence. Absent further details on the purpose and design of the study proposed by the OP, I don't see that much is to be gained by further elaboration; the importance of taking a representative sample from a defined population is a pretty basic idea. $\endgroup$ – EdM Dec 28 '19 at 15:43
  • $\begingroup$ Great additions! (+1) $\endgroup$ – usεr11852 Dec 28 '19 at 22:00
  • $\begingroup$ Thank you for the response. My ability to get more data is limited. I can get an fixed (quite low) number of samples, which practically forces me to oversample the disease cases. I can't see a way to avoid it as the disease itself is quite rare. Maybe it would be wiser to approach it as a case control study and aim for odds ratio instead of risk ratio goal. $\endgroup$ – Xyand Dec 30 '19 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.