In textbooks I always read that it is necessary to have a proper prior on the parameter that we want to test with Bayes factor, otherwise we would always posteriori favor the model with less parameters.

Why is this the case?

  • 1
    $\begingroup$ Hi John, thanks for your question. Can you please clarify the points that we brought up in the comments to @RobinRyder's answer? We're confused about whether you mean to ask about improper/flat priors or just very wide/"diffuse" priors (that are still proper). $\endgroup$ – Jake Westfall Oct 5 '18 at 16:20

The small model isn't necessarily favoured; rather, if you use an improper prior, the Bayes factor is not uniquely defined, because an improper prior is defined up to a multiplicative constant.

The Bayes factor is the ratio of the marginal likelihoods of your two models. With prior $\pi$ and likelihood $L$, you can write the marginal likelihood of a model as $$m(y)=\int \pi(\theta) L(\theta;x)d\theta.$$

If your prior is improper, nothing stops you from replacing $\pi(\theta)$ by $K\cdot \pi(\theta)$, thus multiplying the marginal likelihood by an arbitrary constant $K$. Thus your Bayes factor can take any value in $\mathbb R_+$.

This doesn't happen with proper priors: since they must integrate to 1, you cannot multiply them by $K$.

| cite | improve this answer | |
  • $\begingroup$ I don't think the question was about using an improper prior, but just a very wide/diffuse prior. $\endgroup$ – Jake Westfall Oct 5 '18 at 16:11
  • $\begingroup$ @JakeWestfall - the OP does say "proper prior", not "diffuse prior". $\endgroup$ – jbowman Oct 5 '18 at 16:12
  • $\begingroup$ @jbowman I guess it's ambiguous: the title says "diffuse" but the actual text talks about a "proper" prior. Certainly the conclusion that the OP mentioned is only valid in the case of wide/diffuse priors and not improper priors (as Robin rightly points out), which makes me think they mean to ask about wide/diffuse priors. But the OP should clarify. $\endgroup$ – Jake Westfall Oct 5 '18 at 16:17
  • $\begingroup$ @JakeWestfall - good points, both. $\endgroup$ – jbowman Oct 5 '18 at 16:18
  • $\begingroup$ @JakeWestfall Ah yes, thanks for pointing that out. I focused on the text of the question; I hadn't noticed the discrepancy between the title and the text. You are right that my answer doesn't apply to a diffuse prior, only to an improper prior. $\endgroup$ – Robin Ryder Oct 5 '18 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.