In Orwin's fail safe N test how to decide the values of criterion for a trivial log odd's ratio and mean log odds ratio in missing studies. I am a medical doctor. Please tell me in simple english. The data are

1. Classic fail-safe N 
Z-value for observed studies                                    27.97543
P-value for observed studies                                    0.00000
Alpha                                                          0.05000
Tails                                                           2.00000 
Z for alpha                                                    1.95996
Number of observed studies                                    5.00000
Number of missing studies that wouldbring p-value to >alpha  1014.0000

2. Orwin's fail-safe N
Odds ration in observed studies                              5.7339 
Criterian for a ‘trivial’ odds ratio                           ?
Mean odds ratio in missing studies                           ?
  • $\begingroup$ something is missing in your first sentence. As it stands now it is incomprehensible from linguistic point of view, for me at least. $\endgroup$ – mpiktas Feb 18 '11 at 8:25
  • $\begingroup$ @Bernd: Thank you for your interest to help me. I am sorry I was not clear and wasted your time. I am trying to calculate Orwin's fail-safe N. The treatment is inexpensive but has saved thousands of lives already.I am trying to decide on the best values to use for the criterion for a trivial log odd's ratio and mean log odds ratio in missing studies to calculate the Number of missing studies needed to bring down odds ratio (because Classic fail-safe method yielded 1014 studies). I am using Comprehensive meta-analysis v2.2 $\endgroup$ – DrWho Feb 18 '11 at 13:30

The criterion for a 'trivial' effect size (odds ratio in your example) should be decided based on the size of effect that would be considered 'trivial' in the particular scenario, rather than on statistical grounds. If you're looking at an intervention that may be given to a considerable segment of the population with few side-effects and may prevent early death in a few (statins are one example that come to my mind, but you're the medic), then a small reduction in death rates might still be important, so a trivial reduction could perhaps be 1% or less, i.e. an odds ratio of 0.99 or closer to 1. If you're looking at an invasive or costly intervention or one with severe side-effects, or a condition that is an irritation or of short duration, the trivial reduction would be very much larger.

Rosenthal's original fail-safe N based on statistical significance assumed the mean effect size in missing studies was the null effect size. Orwin's method allows you to choose this, but the null effect size remains the simplest choice.

Having said all that, I don't like either Rosenthal's or Orwin's 'fail-safe N' myself (though I prefer Orwin's to Rosenthal's). As Rosenberg points out in the abstract of the paper below, they "are unweighted and are not based on the framework in which most meta-analyses are performed". He suggests a general, weighted fail-safe N using either the fixed- or random-effects frameworks that are far more commonly used for meta-analysis.

Michael S. Rosenberg. The file-drawer problem revisited: a general weighted method for calculating fail-safe numbers in meta-analysis. Evolution 59 (2):464-468, 2005.

  • $\begingroup$ Thank you so much for the unexpectedly fast and simple worded guidance.You must be an excellent teacher. My treatment is very simple and has already saved thousands of lives. $\endgroup$ – DrWho Feb 18 '11 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.