3
$\begingroup$

I'm interested in references for running empirical Bayes (EB) in conjunction with MCMC. The closest thing I've found to what I'm looking at is a surprisingly recent and somewhat obscure paper available here, and seems to suggest an improved version of the obvious thing to do (do some kind of stochastic gradient descent based on draws from the Markov chain); the main reference to EB applications given in that paper is to a Casella Biostatistics paper (2001) that suggests doing something unreasonable. There is also another surprisingly recent Annals paper that does empirical Bayes by attempting to optimize Bayes factors by running multiple chains (this seems to me like overkill, but to be fair I think this isn't the entire point of what they are doing).

Given how long EB has been studied, there's no way there isn't a completely standard solution for doing EB within MCMC that isn't much older than these papers, right? It seems to me like unless you are dealing with a toy problem you would have to deal with this, and it is an old problem, so there should be less obscure papers.

Improve this question
$\endgroup$
2
  • $\begingroup$ @Ret I copied wrong URL of paper. I linked to authors we page to get around pay wall issues but linked to the wrong paper by mistake. $\endgroup$ – guy Jul 30 '13 at 16:11
  • $\begingroup$ Have you found more references? $\endgroup$ – Q_Li Jan 30 '17 at 4:01
2
$\begingroup$

I do not know much about this topic, but since I found your references very interesting, I felt obliged to give something in return. It appears to me that a reasonable starting point is:

Carlin, Bradley P.; Louis, Thomas A. (2000). Bayes and Empirical Bayes Methods for Data Analysis (2nd ed.). Chapman & Hall/CRC.

For instance, they mention the following in the Preface

A principal reason for the ongoing expansion in the Bayes and EB statistical presence is of course the corresponding expansion in readily-available computing power, and the simultaneous development in Markov chain Monte Carlo (MCMC) methods and software for harnessing it. Our desire to incorporate many recent developments in this area provided one main impetus for this second edition. For example, we now include discussions of reversible jump MCMC, slice sampling, structured MCMC, and other new computing methods and software packages.

Best.

Improve this answer
$\endgroup$
2
  • $\begingroup$ @guy Good, thanks for letting me know. Recall that a $\epsilon$−final version of the papers from IMS is always posted on arXiv: arxiv.org/abs/1202.51600. $\endgroup$ – Ret Jul 30 '13 at 16:19
  • $\begingroup$ Just wanted to give an update. G-modeling, introduced by Brad Efron, is a faster EB implementation than MCMC offers. He fits a spline to the marginal distribution. See here, statweb.stanford.edu/~ckirby/brad/papers/… $\endgroup$ – Alex May 24 '20 at 13:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.