The trouble with meta-analysing a small number of (homogeneous) single-case studies

Question

I am considering doing a meta-analysis on the reported effects of a particular therapy. I've seen it suggested that the minimum number of studies (so far I identified k=7 of them) is not in itself a hard limitation.

However, what I am more concerned about is that most of the studies that were published with this particular therapy, mostly contain single cases, which brings the total sample size across studies to a number only a little higher than the number of studies per se.

I have several doubts/questions:

1) Is overall sample size something to be taken into consideration separately from the number of studies, or are they both just inputs to the same power calculation, as per the paper cited above?

2) How does the single-case nature of some of the to-be-reviewed studies change the methodology that I would have to adopt for the meta-analysis, if they are to be combined with multi-subject studies that do population-level inference? For instance, this older study claims single-case primary studies cannot be included because "mathematics for combining multiple-subject and single-subject outcomes do not exist", whereas Jackson et al's paper from Joe_74's answer below does provide such methods!

3) Is the fact that most of the studies available are carried out by a single lab as opposed to multiple labs, a problem that can be corrected for in the statistics, or does it bias the meta-analysis in ways that are difficult to measure quantitatively?

4) Is a study's lack of a control group reason-enough for exclusion? For instance, I've encountered Cochrane reviews that justify excluding some trials because they were not a randomised controlled trial (RCT) or a controlled clinical trial (CCT)!

Answer 1

The issue of whether, when and how to perform a systematic review and meta-analysis of single case reports is an interesting one.

For further guidance, you may refer to Sampayo-Cordero et al, Murad et al, and Jackson et al, just to name of few interesting references.

Your scenario is however somewhat different, in the sense that you have both aggregate data from some studies and then some case reports. Indeed, you cannot generate estimates of precision unless you have repeated measures from the same subject reported in a given case report.

Yet, pragmatically, you may simply generate another hypothetical study (which we may call study X) which combines the different case reports. Such study X will be then easy to manipulate for inference with the other published studies.

However, as also stated below, consider that only very weak (at best hypothesis generating) evidence can stem from small studies and this is even truer for case reports, as external validity will be minimal.

Regarding your questions:

1) The larger the sample of each study and the larger the overall sample of the meta-analysis the better, but what matter most are precision (within-study and overall) and between-study consistency.

2) I would recommend as stated above to cumulate all single reports as a single hypothetical study X to be analyzed together with the other published studies.

3) This is a key limitation. You may still procede but the potential bias cannot be adjusted altogether. However, plan a subgroup analysis according to institution.

4) It depends on what is your endpoint and your outcome. Single-arm studies are common for epidemiologic assessments (eg prevalence, incidence, proportion), but of course their informativeness for comparative effectiveness research of therapeutic interventions is quite limited.