I am familiar with the mechanics with both methods, but don't know what factors I should consider when choosing between these two approaches for adjusting a prior.

I would imagine that, on a case by case basis, one could run posterior predictive checks, compare model complexity, etc. to see which model gives the best simplicity-fitting tradeoff.

But in the general case, how does one make the decision of whether to use non-informative priors vs Empirical Bayes?


If those are your only two options, then you should use empirical Bayes if and only if you have a hierarchical model. A hierarchical model is when you have multiple parameters $\theta_i$ that are assumed to come from the same prior distribution $p(\theta)$. In this situation, empirical Bayes is a safer choice than setting $p(\theta)$ to be a non-informative prior. If you don't have a hierarchical model, then you can't sensibly learn the prior from data, so the non-informative prior is the better option.

  • $\begingroup$ Thank you Tom. That is certainly my case (having a hierarchical model). Aside from these two options, MAP-II (i.e. MAP on the the hyperparameters using a prior on the priors) and a full Bayes treatment, is there any other way to adjust or inform the choice of the prior? $\endgroup$ – Amelio Vazquez-Reina Sep 12 '14 at 18:25
  • 1
    $\begingroup$ Empirical Bayes is only one of the possible approximations to a full Bayes treatment. There are many other approximation algorithms available, such as Variational Bayes and Expectation Propagation. $\endgroup$ – Tom Minka Sep 12 '14 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.