13
$\begingroup$

On their wiki page, the developers of Stan state:

Some principles we don't like: invariance, Jeffreys, entropy

Instead, I see a lot of normal distribution recommendation. So far I used Bayesian methods that didn't rely on sampling, and was kind of happy to have understood why $\theta \sim \text{Beta}\left(\alpha=\frac{1}{2},\beta=\frac{1}{2}\right)$ was a good choice for binomial likelihoods.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Generic comment: Software documentation doesn't always recapitulate the statistical arguments for what software does and doesn't do. That's true of most R packages I've looked at and I am not surprised to hear the same of Stan. Andrew Gelman is, evidently, a prolific author. $\endgroup$
    – Nick Cox
    Commented Feb 25, 2018 at 9:34
  • 1
    $\begingroup$ Further generic comment: I don't find this kind of question very satisfying, partly because it is about particular individuals. If live authors don't explain somewhere, and are not obviously active here, then send them an email to ask. It is more satisfying to ask in abstraction about the relative merits of different approaches. Sometimes it is fair to say just that you can always use different software if you find something missing, including writing your own. Non-disclosure: never used Stan. $\endgroup$
    – Nick Cox
    Commented Feb 25, 2018 at 9:47
  • $\begingroup$ @NickCox I don't think this question would have benefitted from an anonymization, because (1) the context of a samling software is important (2) my impression is that a rejection of Jeffreys priors is unusual enough that it is worthwhile to point out that a renown source makes that claim. (3) I don't think it is confrontative to cite someone in a question. $\endgroup$
    – wirrbel
    Commented Feb 25, 2018 at 12:17
  • 1
    $\begingroup$ Andy wrote the "Some principles we don't like: invariance, Jeffreys, entropy" but to see why you should look in his book $\endgroup$
    – Ben Goodrich
    Commented Feb 25, 2018 at 16:54
  • 1
    $\begingroup$ Also, this paper contains the most recent thinking on priors among three Stan developers. $\endgroup$
    – Ben Goodrich
    Commented Feb 25, 2018 at 19:23

2 Answers 2

Reset to default
9
$\begingroup$

They do not provide any scientific/mathematical justification for doing so. Most of the developers do not work on this kind of priors, and they prefer to use more pragmatic/heuristic priors, such as normal priors with large variances (which may be informative in some cases). However, it is a bit strange that they are happy to use PC priors, which are based on Entropy (KL divergence), after they started working on this topic.

A similar phenomenon happened with WinBUGS, when the developers recommended the $Gamma(0.001,0.001)$ as a non-informative prior for precision parameters since it resembles the shape of the Jeffreys prior. This prior became the default prior for precision parameters. Later, it was shown (by Gelman!) that they can be highly informative.

Improve this answer
$\endgroup$
2
  • $\begingroup$ could you provide an informative hyperlink/source w.r.t. the Gelman claim. $\endgroup$
    – Jim
    Commented Feb 25, 2018 at 10:03
  • $\begingroup$ @Jim Sure, it is the paper: projecteuclid.org/euclid.ba/1340371048 $\endgroup$
    – Prior
    Commented Feb 25, 2018 at 10:11
14
$\begingroup$

This is of course a diverse set of people with a range of opinions getting together and writing a wiki. I summarize I know/understand with some commentary:

  • Choosing your prior based on computational convenience is an insufficient rationale. E.g. using a Beta(1/2, 1/2) solely because it allows conjugate updating is not a good idea. Of course, once you conclude that it has good properties for the type of problem you work on, that's fine and you might just as well make a choice that makes implementation easy. There's plenty of examples, where convenient default choices turn out to be problematic (see Gamna(0.001, 0.001) prior that enables Gibbs sampling).

  • With Stan - unlike with WinBUGS or JAGS - there is no particular advantage to (conditionally-)conjugate priors. So you might just a well ignore the computational aspect somewhat. Not entirely though, because with very heavy tailed priors (or improper priors) and data that does not identify the parameters well, you run into problems (not really a Stan specific problem, but Stan is quite good at identifying these issues and warning the user instead of happily sampling away).

  • Jeffreys's and other "low information" priors can sometimes be improper or be a bit hard too understand in high dimensions (never mind to derive them) and with sparse data. It may just be that these caused trouble too often for the authors to never comfortable with them. Once you work in something you learn more and get comfortable, hence the occasional opinion reversal.

  • In the sparse data setting the prior really matters and if you can specify that totally implausible values for a parameter are implausible, this helps a lot. This motivates the idea of weakly-informative priors - not truly fully informative priors, but ones with most support for plausible values.

  • In fact, you could wonder why one bothers with uninformative priors, if we have lots of data that identifies the parameters really well (one could just use maximum likelihood). Of course, there's plenty of reasons (avoiding pathologies, getting the "real shape" of posteriors etc.), but in "plenty of data" situations there seems to be no real argument against weakly informative priors instead.

  • Perhaps slightly oddly a N(0, 1) is a surprisingly decent prior for coefficient in logistic, Poisson or Cox regression for many applications. E.g. that's very approximately the distribution of observed treatment effects across a lot of clinical trials.
Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for the detailed answer. I guess my astonishment is not so much about conjugacy (because if I understand this correctly, Jeffreys priors dont need to be conjugate priors, they just need to be invariant under reparametrization). So I would totally understand advice against conjugate priors. $\endgroup$
    – wirrbel
    Commented Feb 25, 2018 at 19:12
  • $\begingroup$ I think the worry with Jeffreys prior is mostly that it is some high dimensional prior that may not be a proper prior and may have some influence on your inference that you do not fully understand. I think that's mostly a concern with sparse data, although perhaps someone can point out an example with non-sparse data, where some problems occur (I'm not aware of any). Plus with Jeffreys prior and various other "uninformative" options, there is the incovenience of actually having to derive it. $\endgroup$
    – Björn
    Commented Feb 26, 2018 at 13:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.