2
$\begingroup$

I am considering doing a meta-analysis on the reported effects of a particular therapy. I've seen it suggested that the minimum number of studies (so far I identified k=7 of them) is not in itself a hard limitation.

However, what I am more concerned about is that most of the studies that were published with this particular therapy, mostly contain single cases, which brings the total sample size across studies to a number only a little higher than the number of studies per se.

I have several doubts/questions:

1) Is overall sample size something to be taken into consideration separately from the number of studies, or are they both just inputs to the same power calculation, as per the paper cited above?

2) How does the single-case nature of some of the to-be-reviewed studies change the methodology that I would have to adopt for the meta-analysis, if they are to be combined with multi-subject studies that do population-level inference? For instance, this older study claims single-case primary studies cannot be included because "mathematics for combining multiple-subject and single-subject outcomes do not exist", whereas Jackson et al's paper from Joe_74's answer below does provide such methods!

3) Is the fact that most of the studies available are carried out by a single lab as opposed to multiple labs, a problem that can be corrected for in the statistics, or does it bias the meta-analysis in ways that are difficult to measure quantitatively?

4) Is a study's lack of a control group reason-enough for exclusion? For instance, I've encountered Cochrane reviews that justify excluding some trials because they were not a randomised controlled trial (RCT) or a controlled clinical trial (CCT)!

$\endgroup$
  • 1
    $\begingroup$ If you're getting an estimate and a standard error from each study, you can meta-analyze them. That is better for interpretation than interpreting them individual. $\endgroup$ – Jeremy Miles Nov 5 '18 at 18:07
  • 1
    $\begingroup$ But surely there's no standard error from studies that only report data (e.g.percentage improvement) from one individual, no? $\endgroup$ – z8080 Nov 5 '18 at 20:53
  • 1
    $\begingroup$ I'm not sure, depends on the design of the study. How were you planning to do a meta-analysis? Or would you combine the data and essentially do an IPD meta analysis? $\endgroup$ – Jeremy Miles Nov 6 '18 at 0:06
  • 1
    $\begingroup$ Looking at the definition of a IPD meta analysis, no, it doesn't sound like what I wanted to do. Rather, I just want to create an aggregate measure of the mean effect size + its variability across studies, be they with one subjects or more. My question was what standard error would I look for in a study that happens to only present a single case? $\endgroup$ – z8080 Nov 6 '18 at 10:58
2
$\begingroup$

The issue of whether, when and how to perform a systematic review and meta-analysis of single case reports is an interesting one.

For further guidance, you may refer to Sampayo-Cordero et al, Murad et al, and Jackson et al, just to name of few interesting references.

Your scenario is however somewhat different, in the sense that you have both aggregate data from some studies and then some case reports. Indeed, you cannot generate estimates of precision unless you have repeated measures from the same subject reported in a given case report.

Yet, pragmatically, you may simply generate another hypothetical study (which we may call study X) which combines the different case reports. Such study X will be then easy to manipulate for inference with the other published studies.

However, as also stated below, consider that only very weak (at best hypothesis generating) evidence can stem from small studies and this is even truer for case reports, as external validity will be minimal.

Regarding your questions:

1) The larger the sample of each study and the larger the overall sample of the meta-analysis the better, but what matter most are precision (within-study and overall) and between-study consistency.

2) I would recommend as stated above to cumulate all single reports as a single hypothetical study X to be analyzed together with the other published studies.

3) This is a key limitation. You may still procede but the potential bias cannot be adjusted altogether. However, plan a subgroup analysis according to institution.

4) It depends on what is your endpoint and your outcome. Single-arm studies are common for epidemiologic assessments (eg prevalence, incidence, proportion), but of course their informativeness for comparative effectiveness research of therapeutic interventions is quite limited.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.