I am looking for a prior for a scale parameter for which I have prior knowledge such that: "$\sigma$ typically does not exceed 100." ("typically" meaning that occasionnally this can happen).

In such a context, I notice in the paper "Prior distributions for variance parameters in hierarchical models" of Andrew Gelman the following recommandation:

[...] When more prior information is desired, for instance to restrict σ away from very large values, we recommend working within the half-t family of prior distributions, which are more flexible and have better behavior near 0, compared to the inverse-gamma family. A reasonable starting point is the half-Cauchy family, with scale set to a value that is high but not off the scale."

As I understand it, a Cauchy (thus half-Cauchy) distribution has an infinite variance and I am not confortable with the idea of building an informative prior with an infinite variance density. Have you some insight on why my interpretation is bad/unsuited ? Moreover, have you some alternative proposals for my prior ?

  • $\begingroup$ "I am not confortable with the idea of building an informative prior with an infinite variance density" -- why is that, exactly? $\endgroup$ – Glen_b Nov 12 '13 at 9:06
  • $\begingroup$ @Glen_b maybe because I wrongly relate informativeness and variance ... When using Gaussian densities for informative priors, roughly speaking the smaller the variance the larger its informativness... $\endgroup$ – peuhp Nov 12 '13 at 9:16
  • 2
    $\begingroup$ If you have information (which is about the only reason I'd use an informative prior in this situation), then use it to construct the prior. Alternatively, if you don't have the information to construct a finite variance prior, why would you insist on an informative one? $\endgroup$ – Glen_b Nov 12 '13 at 9:19
  • 1
    $\begingroup$ Loosely, an informative prior expresses specific, definite information about a variable, while an uninformative prior expresses vague or general information about a variable. It's possible for the half-cauchy to be either, actually, but Gelman's purpose in raising it is to suggest an uninformative (or at best a very weakly informative prior). My point was not to make a specific claim about the half-Cauchy (though if I were to use it, it would be because I was trying to be relatively uninformative); the point was to say that if you really have information, you can construct a prior from it. $\endgroup$ – Glen_b Nov 12 '13 at 9:28
  • 1
    $\begingroup$ You could also truncate the half-Cauchy at some arbitrarily very, very large number, much greater than any conceivable variance for your problem. Then it has moments of all order. Hardly seems worth the effort, though, since the truncated and untruncated Cauchy shapes will be identical over any region of interest, so so will the posteriors. $\endgroup$ – jbowman Nov 12 '13 at 15:31

An alternative to using a Half-Cauchy distribution with a well-defined variance is a Half-Student-t with $\nu>2$ degrees of freedom, e.g. $\nu=3$.

$$\pi(\nu)= \frac{12 \sqrt{3}}{\pi \left(x^2+3\right)^2},\,\,\, \nu>0. $$

This prior has semi-heavy tails and it should produce fairly similar results as the Half-Cauchy prior. You can visualise it in R with the following code curve(2*dt(x,df=3),0,10). It can also be interpreted as "I have prior information, but not much" since you are using something that resembles a "vague prior" but you are adding a bit of information since you think that the tails shouldn't be that heavy. The mass cumulated on $(100,\infty)$ is $1.102261e-06$.

  • $\begingroup$ Thanks! Have you any references about works using such a prior form? $\endgroup$ – peuhp Nov 26 '13 at 20:06
  • $\begingroup$ @peuhp It can be seen as the prior in Gelman (2006) with a Dirac delta prior $\delta_3(\nu)$ on the degrees of freedom. The interpretation is sort of clear, I guess (and I hope). Then, you shouldn't need a reference written by a sacred cow to use it and justify it. $\endgroup$ – Teco Nov 26 '13 at 20:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.