Questions tagged [meta-analysis]

Methods focused on contrasting and combining results from different studies, in the hope of increasing precision and external validity.

Filter by
Sorted by
Tagged with
32
votes
3answers
3k views

Why does finding small effects in large studies indicate publication bias?

Several methodological papers (e.g. Egger et al 1997a, 1997b) discuss publication bias as revealed by meta-analyses, using funnel plots such as the one below. The 1997b paper goes on to say that "...
29
votes
3answers
3k views

Can a meta-analysis of studies which are all “not statistically signficant” lead to a “significant” conclusion?

A meta-analysis includes a bunch of studies, all of which reported a P value greater than 0.05. Is it possible for the overall meta-analysis to report a P value less than 0.05? Under what ...
27
votes
3answers
2k views

What causes the discontinuity in the distribution of published p-values at p < .05?

In a recent paper , Masicampo and Lalande (M-L) collected a large number of p-values published in many different studies. They observed a curious jump in the histogram of the p-values right at the ...
23
votes
2answers
600 views

Statistical forensics: Benford and beyond

What broad methods are there to detect fraud, anomalies, fudging, etc. in scientific works produced by a third party? (I was motivated to ask this by the recent Marc Hauser affair.) Usually for ...
21
votes
2answers
2k views

Combining information from multiple studies to estimate the mean and variance of normally distributed data - Bayesian vs meta-analytic approaches

I have reviewed a set of papers, each reporting the observed mean and SD of a measurement of $X$ in its respective sample of known size, $n$. I want to make the best possible guess about the likely ...
18
votes
5answers
2k views

Does Bayesian statistics make meta-analysis obsolete?

I'm just wondering if Bayesian statistics would be applied consequently from the first study to the last if this makes a meta-analysis obsolete. For example, let's assume 20 studies which have been ...
18
votes
2answers
11k views

Quality assurance and quality control (QA/QC) guidelines for a database

Background I am overseeing the input of data from primary literature into a database. The data entry process is error prone, particularly because users must interpret experimental design, extract ...
17
votes
2answers
12k views

Multiple imputation for outcome variables

I've got a dataset on agricultural trials. My response variable is a response ratio: log(treatment/control). I'm interested in what mediates the difference, so I'm running RE meta-regressions (...
17
votes
4answers
13k views

Combining two confidence intervals/point estimates

Suppose one has two independent samples from the same population, and different methods were used on the two samples to derive point estimate and confidence intervals. In trivial cases a sensible ...
15
votes
4answers
1k views

Looking for good introductory treatment of meta-analysis

A (non-statistician) colleague has been encountering meta-analyses in papers he reviews for medical journals and is looking for a good introductory level treatment so he can educate himself. Any ...
14
votes
3answers
13k views

Pros and cons of meta-analyses

I have been considering doing some meta-analysis for a particular field of study in evolution, but before I go any further I would like to know; what are the positives and negatives of the process? ...
13
votes
2answers
433 views

Without multi-level modeling, how to handle within-study replication in a meta-analysis, where the study is the unit of replication?

Description of the study: I have observed a common error among meta-analyses, with regard to handling of within-study replication. It is not clear to me if the error invalidates the studies when ...
13
votes
2answers
330 views

Making a Bayesian prior from a frequentist result

How should one go about turning a frequentist result into a Bayesian prior? Consider the following pretty generic scenario: An experiment was conducted in the past and a result on some parameter $\...
13
votes
1answer
427 views

Specifying prior for effect size in meta-analysis

My question concerns priors on effect sizes, in my project the measure is Cohen's $D$. Through reading the literature, it seems vague priors are often used, such as in the well-know eight schools ...
12
votes
1answer
332 views

Is meta-analysis of odds ratios essentially hopeless?

In a recent paper Norton et al. (2018)$^{[1]}$ state that Different odds ratios from the same study cannot be compared when the statistical models that result in odds ratio estimates have ...
12
votes
2answers
6k views

How to parameterize the ratio of two normally distributed variables, or the inverse of one?

Problem: I am parameterizing distributions for use as a priors and data in a Bayesian meta-analysis. The data are provided in the literature as summary statistics, almost exclusively assumed to be ...
11
votes
1answer
4k views

Does transformation of r into Fisher z benefit a meta-analysis?

Usually $r$ is transformed into Fisher $z$ to test difference between two $r$ values. But, when a meta-analysis is to be performed, why we should take such a step? Does it correct for measurement ...
11
votes
2answers
1k views

Can I include an effect size as an independent variable in a meta-regression?

My question is whether I can use an effect size $X$ as a dependent variable and another effect size $Y$ as the independent variable in a meta-regression? For example, I conducted a meta-analysis for ...
11
votes
2answers
1k views

Why not perform meta-analysis on partially simulated data?

Background: A typical meta-analysis in psychology might seek to model the correlation between two variables X and Y. The analysis would typically involve obtaining a set of relevant correlations ...
11
votes
3answers
3k views

Are these formulas for transforming P, LSD, MSD, HSD, CI, to SE as an exact or inflated/conservative estimate of $\hat{\sigma}$ correct?

Background I am conducting a meta-analysis that includes previously published data. Often, differences between treatments are reported with P-values, least significant differences (LSD), and other ...
10
votes
2answers
2k views

Justifications for a fixed-effects vs random-effects model in meta-analysis

I have read several publications attempting to justify the use of a fixed effects model with statements along the lines of "the fixed-effects model was chosen because the heterogeneity was low". ...
10
votes
1answer
1k views

Alternative funnel plot, without using standard error (SE)

Before submission of my meta-analysis I want to make a funnel plot to test for heterogeneity and publication bias. I have the pooled effect size and the effect sizes from each study, that take values ...
10
votes
2answers
1k views

Is precision-based (i.e. inverse-variance) weighting integral to meta-analysis?

Is precision-based weighting central to meta-analysis? Borenstein et al. (2009) write that for meta-analysis to be possible all that is necessary is that: Studies report a point estimate which can ...
10
votes
1answer
3k views

Meta-analysis in R using metafor package

How should I syntax the rma function from metafor package in order to get results in the following real-life example of a small meta-analysis? (random-effect, ...
10
votes
1answer
600 views

Torn between PET-PEESE and multilevel approaches to meta-analysis: is there a happy medium?

I am currently working on a meta-analysis, for which I need to analyze multiple effect sizes nested within samples. I am partial to Cheung's (2014) three-level meta-analysis approach to meta-analyzing ...
10
votes
1answer
347 views

So how would you include Bayesian estimates in a meta-analysis?

Inspired by this question, and particular "Problem 3": Posterior distributions are somewhat more difficult to incorporate into a meta-analysis, unless a frequentist, parametric description of the ...
9
votes
1answer
13k views

How to calculate Standard Error of Odds Ratios?

I have two datasets from genome-wide association studies. The only information available is the odds ratio and the p-value for the first data set. For the second data set I have the Odds Ratio, p-...
9
votes
4answers
11k views

How to calculate confidence intervals for pooled odd ratios in meta-analysis?

I have two datasets from genome-wide association studies. The only information available are the odd ratios and their confidence intervals (95%) for each genotyped SNP. My want to generate a forest ...
9
votes
2answers
492 views

How to best handle subscores in a meta-analysis?

I am conducting a meta-analysis of effect sizes d in R using the metafor package. d represents differences in memory scores between patients and healthy. However some studies report only subscores of ...
9
votes
3answers
2k views

Meta analysis on studies with 0-frequency cells

I am familiar with meta analysis and meta regression techniques (using the R package metafor from Viechtbauer), but I recently stumbled on a problem I can't easily ...
9
votes
1answer
2k views

Metafor package: bias and sensitivity diagnostics

I am conducting a multi-level meta-analysis that includes some articles with multiple outcomes. Therefore I am using the rma.mv() function. Example code: ...
9
votes
2answers
530 views

Alternate weighting schemes for random effects meta-analysis: missing standard deviations

I am working on a random effects meta-analysis covering a number of studies which do not report standard deviations; all studies do report sample size. I do not believe it is possible to approximate ...
9
votes
1answer
319 views

Question about inverse-variance weighting

Suppose we want to make inference on an unobserved realization $x$ of a random variable $\tilde x$, which is normally distributed with mean $\mu_x$ and variance $\sigma^2_x$. Suppose there is another ...
9
votes
1answer
1k views

What is the difference between a Frequentist approach with meta-analysis and a Bayesian approach?

Say I am performing an analysis looking at a particular health measure. I am interested in the difference in that measure between patients and controls and whether or not the difference is different ...
8
votes
2answers
2k views

In a meta-analysis, how should one handle non-significant studies containing no raw data?

Let's say that I'm conducting a meta-analysis, looking at the performance of group A and group B with respect to a certain construct. Now, some of the studies that I'll come across will report that no ...
8
votes
3answers
3k views

Calculating effect sizes and standard errors for the difference between two standardized mean differences

I have two related questions, both of which are related to a meta-analysis I am performing where where the primary outcomes are expressed in terms of the standardized mean difference. My studies have ...
8
votes
1answer
2k views

Oddly large R squared values in meta regression (metafor)

I am using the metafor package in R. I have fit a random effects model with a continuous predictor as follows SIZE=rma(yi=Ds,sei=SE,data=VPPOOLed,mods=~SIZE) ...
8
votes
1answer
2k views

Multilevel multivariate meta-regression

Background: I'd like to conduct a meta-regression using studies which have (1) several outcomes/constructs (= multivariate) and (2) multiple effect sizes for every of these outcomes because of ...
8
votes
2answers
219 views

Causes of bimodal distributions when bootstrapping a meta-analysis model

I help a colleague to bootstrap a meta-analysis mixed-effects model using the metafor R package framework authored by @Wolfgang. Interestingly and worryingly, for one of the model's coefficients I ...
8
votes
1answer
6k views

How to conduct a meta-analysis on studies that report results variously as odds ratios, hazards ratios, or rate ratios?

I'm doing a meta-analysis of some studies that report results variously as odds ratios, hazards ratios or rate ratios (all with confidence intervals). Is there any way to combine these together/...
8
votes
4answers
3k views

Why does Stouffer's method work?

It seems like a fairly straightforward question, but when I really think about it, Stouffer's method doesn't make sense to me. This is why: Assume a two-tailed hypothesis. You first calculate $z_i$ ...
8
votes
1answer
18k views

Calculating I-squared (in meta-analysis, given tau-squared)

I have results of meta-analysis, which have been carried out and reported. These analyses report tau-squared as a measure of heterogeneity (and the other details from the analysis). I've been asked to ...
8
votes
3answers
6k views

Convert hazards ratio to odds ratio

In meta-analysis: How do we convert hazard ratios in some studies to odds ratio? There are case control and cohort studies to be included and some of them report hazard ratios. The raw data is not ...
8
votes
1answer
745 views

Combining data from different sources

I want to combine data from different sources. Let's say I want to estimate a chemical property (e.g. a partitioning coefficient): I have some empirical data, varying due to measurement error around ...
8
votes
1answer
138 views

Is there any meaningfully robust approach to conduct a network meta-analysis of diagnostic test accuracy studies?

Background: I am working on a systematic review including several imaging modalities for coronary artery disease, but the evidence network is quite large, including different modalities, often ...
7
votes
1answer
11k views

Converting odds ratios to Cohen's d for meta analysis

I'm conducting a meta-analysis largely on mean differences data, however I have several articles that report only an odds ratio, N, p value, and confidence interval, and I need to convert this ...
7
votes
1answer
2k views

How to choose between fixed-effects and random-effects model in meta-analysis?

In meta-analysis packages, both fixed effects and random effects models are available. How do one choose between these two models? Since one is assessing different studies, should one not choose ...
7
votes
1answer
288 views

Discrepancy between metafor and weighted lm() standard errors

My friend and I are trying to understand why a weighted lm() v. a fixed-effect rma model from the ...