Questions tagged [meta-analysis]
Methods focused on contrasting and combining results from different studies, in the hope of increasing precision and external validity.
879
questions
32
votes
3answers
3k views
Why does finding small effects in large studies indicate publication bias?
Several methodological papers (e.g. Egger et al 1997a, 1997b) discuss publication bias as revealed by meta-analyses, using funnel plots such as the one below.
The 1997b paper goes on to say that "...
29
votes
3answers
3k views
Can a meta-analysis of studies which are all “not statistically signficant” lead to a “significant” conclusion?
A meta-analysis includes a bunch of studies, all of which reported a P value greater than 0.05. Is it possible for the overall meta-analysis to report a P value less than 0.05? Under what ...
27
votes
3answers
2k views
What causes the discontinuity in the distribution of published p-values at p < .05?
In a recent paper , Masicampo and Lalande (M-L) collected a large number of p-values published in many different studies. They observed a curious jump in the histogram of the p-values right at the ...
23
votes
2answers
600 views
Statistical forensics: Benford and beyond
What broad methods are there to detect fraud, anomalies, fudging, etc. in scientific works produced by a third party? (I was motivated to ask this by the recent Marc Hauser affair.) Usually for ...
21
votes
2answers
2k views
Combining information from multiple studies to estimate the mean and variance of normally distributed data - Bayesian vs meta-analytic approaches
I have reviewed a set of papers, each reporting the observed mean and SD of a measurement of $X$ in its respective sample of known size, $n$. I want to make the best possible guess about the likely ...
18
votes
5answers
2k views
Does Bayesian statistics make meta-analysis obsolete?
I'm just wondering if Bayesian statistics would be applied consequently from the first study to the last if this makes a meta-analysis obsolete.
For example, let's assume 20 studies which have been ...
18
votes
2answers
11k views
Quality assurance and quality control (QA/QC) guidelines for a database
Background
I am overseeing the input of data from primary literature into a database. The data entry process is error prone, particularly because users must interpret experimental design, extract ...
17
votes
2answers
12k views
Multiple imputation for outcome variables
I've got a dataset on agricultural trials. My response variable is a response ratio: log(treatment/control). I'm interested in what mediates the difference, so I'm running RE meta-regressions (...
17
votes
4answers
13k views
Combining two confidence intervals/point estimates
Suppose one has two independent samples from the same population, and different methods were used on the two samples to derive point estimate and confidence intervals. In trivial cases a sensible ...
15
votes
4answers
1k views
Looking for good introductory treatment of meta-analysis
A (non-statistician) colleague has been encountering meta-analyses in papers he reviews for medical journals and is looking for a good introductory level treatment so he can educate himself. Any ...
14
votes
3answers
13k views
Pros and cons of meta-analyses
I have been considering doing some meta-analysis for a particular field of study in evolution, but before I go any further I would like to know; what are the positives and negatives of the process? ...
13
votes
2answers
433 views
Without multi-level modeling, how to handle within-study replication in a meta-analysis, where the study is the unit of replication?
Description of the study:
I have observed a common error among meta-analyses, with regard to handling of within-study replication. It is not clear to me if the error invalidates the studies when ...
13
votes
2answers
330 views
Making a Bayesian prior from a frequentist result
How should one go about turning a frequentist result into a Bayesian prior?
Consider the following pretty generic scenario:
An experiment was conducted in the past and a result on some parameter $\...
13
votes
1answer
427 views
Specifying prior for effect size in meta-analysis
My question concerns priors on effect sizes, in my project the measure is Cohen's $D$. Through reading the literature, it seems vague priors are often used, such as in the well-know eight schools ...
12
votes
1answer
332 views
Is meta-analysis of odds ratios essentially hopeless?
In a recent paper Norton et al. (2018)$^{[1]}$ state that
Different odds ratios from the same study cannot be compared when the
statistical models that result in odds ratio estimates have ...
12
votes
2answers
6k views
How to parameterize the ratio of two normally distributed variables, or the inverse of one?
Problem:
I am parameterizing distributions for use as a priors and data in a Bayesian meta-analysis. The data are provided in the literature as summary statistics, almost exclusively assumed to be ...
11
votes
1answer
4k views
Does transformation of r into Fisher z benefit a meta-analysis?
Usually $r$ is transformed into Fisher $z$ to test difference between two $r$ values. But, when a meta-analysis is to be performed, why we should take such a step? Does it correct for measurement ...
11
votes
2answers
1k views
Can I include an effect size as an independent variable in a meta-regression?
My question is whether I can use an effect size $X$ as a dependent variable and another effect size $Y$ as the independent variable in a meta-regression?
For example, I conducted a meta-analysis for ...
11
votes
2answers
1k views
Why not perform meta-analysis on partially simulated data?
Background:
A typical meta-analysis in psychology might seek to model the correlation between two variables X and Y. The analysis would typically involve obtaining a set of relevant correlations ...
11
votes
3answers
3k views
Are these formulas for transforming P, LSD, MSD, HSD, CI, to SE as an exact or inflated/conservative estimate of $\hat{\sigma}$ correct?
Background
I am conducting a meta-analysis that includes previously published data. Often, differences between treatments are reported with P-values, least significant differences (LSD), and other ...
10
votes
2answers
2k views
Justifications for a fixed-effects vs random-effects model in meta-analysis
I have read several publications attempting to justify the use of a fixed effects model with statements along the lines of "the fixed-effects model was chosen because the heterogeneity was low". ...
10
votes
1answer
1k views
Alternative funnel plot, without using standard error (SE)
Before submission of my meta-analysis I want to make a funnel plot to test for heterogeneity and publication bias. I have the pooled effect size and the effect sizes from each study, that take values ...
10
votes
2answers
1k views
Is precision-based (i.e. inverse-variance) weighting integral to meta-analysis?
Is precision-based weighting central to meta-analysis? Borenstein et al. (2009) write that for meta-analysis to be possible all that is necessary is that:
Studies report a point estimate which can ...
10
votes
1answer
3k views
Meta-analysis in R using metafor package
How should I syntax the rma function from metafor package in order to get results in the following real-life example of a small meta-analysis?
(random-effect, ...
10
votes
1answer
600 views
Torn between PET-PEESE and multilevel approaches to meta-analysis: is there a happy medium?
I am currently working on a meta-analysis, for which I need to analyze multiple effect sizes nested within samples. I am partial to Cheung's (2014) three-level meta-analysis approach to meta-analyzing ...
10
votes
1answer
347 views
So how would you include Bayesian estimates in a meta-analysis?
Inspired by this question, and particular "Problem 3":
Posterior distributions are somewhat more difficult to incorporate into a meta-analysis, unless a frequentist, parametric description of the ...
9
votes
1answer
13k views
How to calculate Standard Error of Odds Ratios?
I have two datasets from genome-wide association studies. The only
information available is the odds ratio and the p-value for the first
data set. For the second data set I have the Odds Ratio, p-...
9
votes
4answers
11k views
How to calculate confidence intervals for pooled odd ratios in meta-analysis?
I have two datasets from genome-wide association studies. The only information available are the odd ratios and their confidence intervals (95%) for each genotyped SNP.
My want to generate a forest ...
9
votes
2answers
492 views
How to best handle subscores in a meta-analysis?
I am conducting a meta-analysis of effect sizes d in R using the metafor package. d represents differences in memory scores between patients and healthy. However some studies report only subscores of ...
9
votes
3answers
2k views
Meta analysis on studies with 0-frequency cells
I am familiar with meta analysis and meta regression techniques (using the R package metafor from Viechtbauer), but I recently stumbled on a problem I can't easily ...
9
votes
1answer
2k views
Metafor package: bias and sensitivity diagnostics
I am conducting a multi-level meta-analysis that includes some articles with multiple outcomes. Therefore I am using the rma.mv() function. Example code:
...
9
votes
2answers
530 views
Alternate weighting schemes for random effects meta-analysis: missing standard deviations
I am working on a random effects meta-analysis covering a number of studies which do not report standard deviations; all studies do report sample size. I do not believe it is possible to approximate ...
9
votes
1answer
319 views
Question about inverse-variance weighting
Suppose we want to make inference on an unobserved realization $x$ of a random variable $\tilde x$, which is normally distributed with mean $\mu_x$ and variance $\sigma^2_x$. Suppose there is another ...
9
votes
1answer
1k views
What is the difference between a Frequentist approach with meta-analysis and a Bayesian approach?
Say I am performing an analysis looking at a particular health measure. I am interested in the difference in that measure between patients and controls and whether or not the difference is different ...
8
votes
2answers
2k views
In a meta-analysis, how should one handle non-significant studies containing no raw data?
Let's say that I'm conducting a meta-analysis, looking at the performance of group A and group B with respect to a certain construct. Now, some of the studies that I'll come across will report that no ...
8
votes
3answers
3k views
Calculating effect sizes and standard errors for the difference between two standardized mean differences
I have two related questions, both of which are related to a meta-analysis I am performing where where the primary outcomes are expressed in terms of the standardized mean difference.
My studies have ...
8
votes
1answer
2k views
Oddly large R squared values in meta regression (metafor)
I am using the metafor package in R. I have fit a random effects model with a continuous predictor as follows
SIZE=rma(yi=Ds,sei=SE,data=VPPOOLed,mods=~SIZE)
...
8
votes
1answer
2k views
Multilevel multivariate meta-regression
Background:
I'd like to conduct a meta-regression using studies which have (1) several outcomes/constructs (= multivariate) and (2) multiple effect sizes for every of these outcomes because of ...
8
votes
2answers
219 views
Causes of bimodal distributions when bootstrapping a meta-analysis model
I help a colleague to bootstrap a meta-analysis mixed-effects model using the metafor R package framework authored by @Wolfgang.
Interestingly and worryingly, for one of the model's coefficients I ...
8
votes
1answer
6k views
How to conduct a meta-analysis on studies that report results variously as odds ratios, hazards ratios, or rate ratios?
I'm doing a meta-analysis of some studies that report results variously as odds ratios, hazards ratios or rate ratios (all with confidence intervals). Is there any way to combine these together/...
8
votes
4answers
3k views
Why does Stouffer's method work?
It seems like a fairly straightforward question, but when I really think about it, Stouffer's method doesn't make sense to me. This is why:
Assume a two-tailed hypothesis. You first calculate $z_i$ ...
8
votes
1answer
18k views
Calculating I-squared (in meta-analysis, given tau-squared)
I have results of meta-analysis, which have been carried out and reported. These analyses report tau-squared as a measure of heterogeneity (and the other details from the analysis). I've been asked to ...
8
votes
3answers
6k views
Convert hazards ratio to odds ratio
In meta-analysis: How do we convert hazard ratios in some studies to odds ratio? There are case control and cohort studies to be included and some of them report hazard ratios. The raw data is not ...
8
votes
1answer
745 views
Combining data from different sources
I want to combine data from different sources.
Let's say I want to estimate a chemical property (e.g. a partitioning coefficient):
I have some empirical data, varying due to measurement error around ...
8
votes
1answer
138 views
Is there any meaningfully robust approach to conduct a network meta-analysis of diagnostic test accuracy studies?
Background: I am working on a systematic review including several imaging modalities for coronary artery disease, but the evidence network is quite large, including different modalities, often ...
7
votes
1answer
11k views
Converting odds ratios to Cohen's d for meta analysis
I'm conducting a meta-analysis largely on mean differences data, however I have several articles that report only an odds ratio, N, p value, and confidence interval, and I need to convert this ...
7
votes
1answer
2k views
How to choose between fixed-effects and random-effects model in meta-analysis?
In meta-analysis packages, both fixed effects and random effects models are available. How do one choose between these two models? Since one is assessing different studies, should one not choose ...
7
votes
1answer
288 views
Discrepancy between metafor and weighted lm() standard errors
My friend and I are trying to understand why a weighted lm() v. a fixed-effect rma model from the ...
