18
$\begingroup$

I'm just wondering if Bayesian statistics would be applied consequently from the first study to the last if this makes a meta-analysis obsolete.

For example, let's assume 20 studies which have been done at different timepoints. The estimate or distribution of the first study was done with a uninformative prior. The second study uses the posterior distribution as the prior. The new posterior distribution is now used as prior for the third study and so on.

At the end we have an estimate which contains all the estimates or data which have been done before. Does it makes sense to do a meta-analysis?

Interestingly, I suppose that changing the order of this analysis would also change the last posterior distribution, respectivly, estimate.

$\endgroup$
14
$\begingroup$

What you are describing is called Bayesian updating. If you can assume that subsequent trials are exchangeable, then it won't matter if you updated your prior sequentially, all at once, or in different order (see e.g. here or here). Notice that if previous experiments influence your future experiments, then also in the case of classical meta-analysis there would be a dependence that is not taken into consideration (if assuming exchangeability).

It makes perfect sense to update your knowledge using Bayesian updating, since it's simply another way of doing it, then using classical meta-analysis. The question if it makes the traditional meta-analysis obsolete, or not, is opinion based and depends if you are willing to adopt Bayesian viewpoint. The most important difference between both approaches is that in Bayesian case you explicitly state your prior assumptions.

$\endgroup$
  • 1
    $\begingroup$ I downvoted this answer, not because it has definitively incorrect, but rather because in regards the question asked by OP, it is very easy to come to the incorrect conclusion. I believe the OP is asking "by doing Bayesian updating, can I disregard fundamental issues with meta-analyses"? It could be easy to misinterpret this answer as "yes, as long as you have no problem with Bayesian analyses". As I point out in my answer, that is not the case. $\endgroup$ – Cliff AB Jan 17 '17 at 21:22
  • 1
    $\begingroup$ @CliffAB I don't think your interpretation of the question is correct. While I upvoted your answer since it brings important issue, I understand the question as asking if Bayesian updating can by used for conducting meta-analysis. My answer is yes it can and I did not state anywhere that when doing so you are approaching problem with ignoring the fundamental rules of meta-analysis. $\endgroup$ – Tim Jan 17 '17 at 21:27
  • $\begingroup$ Perhaps I misreading the OP's intent. But in the following quote "At the end we have an estimates which contains all the estimates which have been done before. Does it makes sense to do a meta-analysis?", the answer should be "Yes!", not "you don't have to if you did Bayesian updating", which I read as what they were implying. $\endgroup$ – Cliff AB Jan 17 '17 at 21:32
  • 1
    $\begingroup$ @CliffAB if sequential analysis (not exactly meta-analysis but something closer to what OP described) was done using Bayesian updating, then all the information - from the prior and from data that appears on subsequent trials - then indeed there is no need for any meta analysis, since you updated your knowledge sequentially and already have your estimate. $\endgroup$ – Tim Jan 17 '17 at 21:35
  • 1
    $\begingroup$ @CliffAB I don't agree with you. It seems that our disagreement is based on the fact that you seem to regard this question as asking about conducting classical meta-analysis. On another hand, as I already stated, I read it as more broad problem, and so my answer is vague and not focusing on any particular data-analytic problem. $\endgroup$ – Tim Jan 17 '17 at 22:15
15
$\begingroup$

I'm sure many people would argue as to what the purpose of a meta-analysis is, but perhaps at a meta-meta level the point of such analysis is to study the studies rather than obtain a pooled parameter estimate. We are interested in whether effects are consistent among each other, of the same direction, have CI bounds that are inversely proportional to the root of the sample size approximately, and so on. Only when all the studies seem to point to the same effect size and magnitude for an association or treatment effect do we tend to report, with some confidence, that what has been observed may be a "truth".

Indeed, there are frequentist ways of conducting a pooled analysis, such as just aggregating evidence from multiple studies with random effects to account for heterogeneity. A Bayesian approach is a nice modification of this, because you can be explicit about how one study might inform another.

Just as well, there are Bayesian approaches to "studying the studies" as a typical (frequentist) meta analysis might do, but that's not what you're describing here.

$\endgroup$
  • 2
    $\begingroup$ Here Is an interesting presentation about Bayesian Meta-Analysis by Chuan Zhou from the Biostatistics Department at Vanderbilt University. Maybe Frank Harrell is familiar with it: biostat.mc,vanderbilt.edu/wiki/pub/Main/BayesianDataAnalysisWithOpenBUGSAndBRugs/BUGSintro_0306.pdf . $\endgroup$ – Michael R. Chernick Jan 14 '17 at 18:19
  • $\begingroup$ I agree that the main concern should be to study the study. Actually, I would also state this is valid for the single study (study the observation). My concern is if the data (estimates, CI, SE) of single studies are partly Bayesian updated can this studies be used for a meta-analysis? $\endgroup$ – giordano Jan 17 '17 at 5:02
  • $\begingroup$ @giordano per your "study the observation" bit, that seems to be the goal with diagnostics. If you have studies whose primary inference comes from Bayesian updating but the studies are still independent of one another, you could use typical meta-analytic approaches (frequentist or approximate Bayesian analogues) remembering that the exact specification of the prior is now one of the many things which may lead to inconsistent findings. If they are not independent then you need to account for that dependence, in a way that may appeal to Bayes Law but not be "Bayesian" per se. $\endgroup$ – AdamO Jan 17 '17 at 16:47
11
$\begingroup$

When one wants to do meta-analysis as opposed to fully prospective research, I view Bayesian methods as allowing one to get more accurate meta-analysis. For example, Bayesian biostatistician David Spiegelhalter showed years ago that the most commonly used method for meta-analysis, the DerSimonian and Laird method, is overconfident. See http://www.citeulike.org/user/harrelfe/article/13264878 for details.

Related to earlier posts when the number of studies is limited I prefer to think of this as Bayesian updating, which allows the posterior distribution from previous studies to be any shape and does not require the assumption of exchangeability. It just requires the assumption of applicability.

$\endgroup$
6
$\begingroup$

One important clarification about this question.

You certainly can do a meta-analysis in the Bayesian settings. But simply using a Bayesian perspective does not allow you to forget about all the things you should be concerned about in a meta-analysis!

Most directly to the point is that good methods for meta-analyses acknowledge that the underlying effects are not necessarily uniform study to study. For example, if you want to combine the mean from two different studies, it is helpful to think about the means as

$\mu_1 = \mu + \alpha_1$

$\mu_2 = \mu + \alpha_2$

$\alpha_1 + \alpha_2 = 0$

where $\mu_1$ is the population mean from study 1, $\mu_2$ is the population mean from study 2, $\mu$ is the global mean of interest, and $\alpha_1$ and $\alpha_2$ are the deviation from the global mean in each study. Of course, you hope that $\alpha_1$ and $\alpha_2$ are very small in magnitude, but assuming 0 is a bit foolish.

This model can easily be fit in a Bayesian framework, just as it could be fit in a frequentist framework. My only point is that in the OP's question, it could be read as using the naive model of assuming $\alpha = 0$ is okay if you are in the Bayesian setting, which is still naive but with a naive prior as well.

So in conclusion, no, Bayesian methods do not make the field of meta-analysis obsolete. Rather, Bayesian methods work nicely hand-in-hand with meta-analyses.

$\endgroup$
5
$\begingroup$

People have tried to analyse what happens when you perform meta-analysis cumulatively although their main concern is to establish whether it is worth collecting more data or conversely whether enough is already enough. For instance Wetterslev and colleagues in J Clin Epid here. The same authors have a number of publications on this theme which are fairly easy to find. I think at least some of them are open access.

$\endgroup$
  • 1
    $\begingroup$ Thanks for the reference. I didn't know about cumulative meta-analysis (CM). I think that cumulative meta-analysis according to this [definition ](bandolier.org.uk/booth/glossary/cumulative.html) is not the same as the inclusion of studies as I stated in my question. In CM each study is a distinct (frequentistic?) study whereas the studies mentioned in my questions contains already the previously studies. $\endgroup$ – giordano Jan 14 '17 at 14:54
  • 3
    $\begingroup$ The paper you are citing refers to sequential clinical trials, e.g. multiple comparisons at time points in the same, single study. The term "meta-analysis" here seems to have a specific meaning not applicable to the OP 's question. $\endgroup$ – AdamO Jan 14 '17 at 17:19
  • $\begingroup$ @AdamO I agree that the use of the phrase "trial sequential analysis" here is misleading but it is directed at meta-analysis and I have certainly reviewed several articles for journals which have used it within their meta-analyses for the purposes I suggested. $\endgroup$ – mdewey Jan 15 '17 at 12:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.