I am working with a student who has collected about 300 participants for his thesis. After collecting 100 participants, he analyzed the data. I had just recently read of a "mini-meta-analysis" strategy where someone collects pilot data, computes the estimates of interest, then performs a larger study and aggregates the estimates later using meta-analysis (Cumming, 2014). I suggested the student do something similar: run the analysis an additional time and use meta-analysis to pool the estimates.
The student asked what would be the advantage of using a meta-analysis relative to just analyzing all 300 participants. I couldn't provide a good answer (unless there was a lot of heterogeneity between the samples, which there isn't).
So, now to my question: if we have the raw data from 2+ homogenous studies, is there an advantage to computing the estimates twice then meta-analyzing them vs. computing them once?
My first impressions are:
The first study allows you to "calibrate" the model, then cross-validate in the second model. However, in this case, his statistical analysis was strictly confirmatory (and no "calibration" was required, aside from computing parameter estimates).
Having two samples allows us to do empirical bayesian analysis (we can use the parameters estimated in sample 1 as priors in sample 2 and take advantage of the strengths of bayesian statistics).
Is there another reason to favor one strategy over another?
Cumming, G. (2014). The new statistics: Why and how. Psychological science, 25(1), 7-29.