Orwin's fail safe N test

Question

In Orwin's fail safe N test how to decide the values of criterion for a trivial log odd's ratio and mean log odds ratio in missing studies. I am a medical doctor. Please tell me in simple english. The data are

1. Classic fail-safe N 
Z-value for observed studies                                    27.97543
P-value for observed studies                                    0.00000
Alpha                                                          0.05000
Tails                                                           2.00000 
Z for alpha                                                    1.95996
Number of observed studies                                    5.00000
Number of missing studies that wouldbring p-value to >alpha  1014.0000

2. Orwin's fail-safe N
Odds ration in observed studies                              5.7339 
Criterian for a ‘trivial’ odds ratio                           ?
Mean odds ratio in missing studies                           ?

Answer 1

The criterion for a 'trivial' effect size (odds ratio in your example) should be decided based on the size of effect that would be considered 'trivial' in the particular scenario, rather than on statistical grounds. If you're looking at an intervention that may be given to a considerable segment of the population with few side-effects and may prevent early death in a few (statins are one example that come to my mind, but you're the medic), then a small reduction in death rates might still be important, so a trivial reduction could perhaps be 1% or less, i.e. an odds ratio of 0.99 or closer to 1. If you're looking at an invasive or costly intervention or one with severe side-effects, or a condition that is an irritation or of short duration, the trivial reduction would be very much larger.

Rosenthal's original fail-safe N based on statistical significance assumed the mean effect size in missing studies was the null effect size. Orwin's method allows you to choose this, but the null effect size remains the simplest choice.

Having said all that, I don't like either Rosenthal's or Orwin's 'fail-safe N' myself (though I prefer Orwin's to Rosenthal's). As Rosenberg points out in the abstract of the paper below, they "are unweighted and are not based on the framework in which most meta-analyses are performed". He suggests a general, weighted fail-safe N using either the fixed- or random-effects frameworks that are far more commonly used for meta-analysis.

Michael S. Rosenberg. The file-drawer problem revisited: a general weighted method for calculating fail-safe numbers in meta-analysis. Evolution 59 (2):464-468, 2005.