Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite

Is it possible to do meta analysis of only two studies. What will be limitation of such analysis.

share|improve this question

2 Answers

up vote 6 down vote

Yes, it is possible, but whether it is appropriate depends on the intent of your analysis.

Meta-analysis is a method of combining information from different sources, so it is technically possible to do a meta-analysis of only two studies - even of multiple results within a single paper. The key concern is not if you can do this, but that the method is appropriate for the questions that you have and the conclusions that you want to make, and that you acknowledgE the limitations of your analysis.

For example, the typical use of meta-analysis is to quantitatively synthesize previous studies on a particular subject, such as the effects of some medical intervention. In this context, it is important to make your criteria for study selection before the analysis and then find all studies available that meet those criteria. These criteria might limit the scope of your search to publications in English, in a particular journal or set of journals, those that use particular methods, etc. In practice, it is necessary to be familiar with the studies you are interested in to state these criteria. However, if you non-randomly select two papers from among many that have been published, it would introduce bias into your study. If only two studies have been published, it might be hard to justify any conclusions from a meta-analysis but it could still be done.

On the other hand, I have used the meta-analytical approach to synthesize data from a single study, for example if summary statistics are reported for subgroups but I am interested in finding the overall mean and variance. I don't always call this a meta-analysis in mixed company, so as not to confuse this application of the method with the more common use of meta-analysis as a comprehensive review sensu stricto.

share|improve this answer
Technical Note: "...However, if you non-randomly select two papers from among many that have been published, it would introduce bias into your study..." This bias you speak of is in the generality of the results to other studies, not in the combination of the two pieces of analysis. The correct analysis will never be biased no matter how the data was obtained, as long as the conclusions are contained to populations sufficiently similar to that which was observed. The potential bias comes in when the population you are inferring about is not sufficiently similar to the sample. – probabilityislogic Jan 23 '11 at 2:01
@probabilityislogic Your point is on the same lines as mine: validity of the analysis is contingent on interpretation. Still, study selection is a special case on which many assumptions of meta-analysis are based. Thes assumptions are too many to review here, but the conclusions would be different if two studies are done on the same population by the same lab vs by two independent labs. The difference is not the population itself but in the investigator, and a common assumption tested by meta-analysis is the absence of an investigator effect. – David Jan 23 '11 at 2:36
up vote 0 down vote

If you compute a likelihood ratio for the effect of interest in each study, you can simply multiply them together to obtain the aggregate weight of evidence for the effect.

share|improve this answer
2  
But what if the two effects have the opposite signs? Likelihood ratios may be an alternative to $p$-values, but I wouldn't recommend meta-analysing $p$-values either unless they're the only thing reported. – onestop Jan 23 '11 at 0:03
@onestop - I'd say the "opposite sign" effect is a direct consequence of using a 2-sided test. In this test, the "null hypothesis" is no effect, whereas the alternative is for some effect, regardless of sign $effect = 0$ vs $effect \neq 0$. If the likelihood is symmetric in the effect, then a change of sign (with no decrease in absolute value) is further evidence in favour of the alternative hypothesis. – probabilityislogic Jan 23 '11 at 2:11
@onestop meta-analysis uses the mean and standard deviation, and $P$ value itself is insufficient since sd can only be estimated from $P$-values if $n$ is also provided, the statistical model is clearly stated. A method called 'vote-counting' tallies $P$-values indicating positive, negative, and no effects, but I am not sure that this is useful. – David Jan 23 '11 at 2:18
@onestop - just adding a further clarification to my comment above. I think why it is intuitively silly for a change of sign to increase the evidence, is that the hypothesis we think of changes between experiments, from $effect=0$ vs $effect \neq 0$ to $effect=E_1$ vs $effect \neq E_1$ where $E_1$ is the estimate from the first study. Shows that one needs to be careful when combining data, to make sure you know which hypothesis test you are doing. – probabilityislogic Jan 23 '11 at 2:22
@onestop - I hadn't considered the issue of different signs in the effects. Good call. – Mike Lawrence Jan 25 '11 at 14:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.