Do researchers ever put Bayesian priors onto different sets of assumptions?

Question

Econometric analyses of causal effects often hinge on assumptions that authors leave unverified ("my model controls for everything important"). Discussions of the analyses often involve equally unverified criticisms ("well, maybe you didn't control for everything important"). This leaves most work in econometrics unconvincing and unsatisfying.

Econometricians are also not very rigorous in the way they quantify the plausibility of their arguments. Mostly Harmless Econometrics, for example, quotes Orley Ashenfelter as saying that the evidence linking education and income is "pretty convincing", and while he makes (what seems to me) a strong case for this assertion, this is presumably as rigorous as he knows how to be about whether evidence is convincing or not. Orley Ashenfelter is also a more careful researcher than most econometricians.

In a field where the best practicioners use quantifiers like "pretty" and "not very" to describe the strength of evidence, and the link between evidence and hypothesis is very uncertain, it seems that authors could make better use of Bayes' Rule: the plausibility of different assumptions could be described using priors, and the strength of evidence for one assumption versus another could be described using likelihoods.

Are there examples of econometricians (or anyone else who needs to reason statistically under uncertain assumptions) using Bayes' Rule in such a way?

Answer 1

Bayesian approach enables you to bring out-of-data information into your model through choice of priors, while likelihood function tells us about information contained in data. Through Bayes theorem

$$ \underbrace{p(\theta \mid X)}_\text{posterior} \propto \underbrace{p(X \mid \theta)}_\text{likelihood} \; \underbrace{p(\theta)}_\text{prior} $$

the two sources of information are combined together. If your data conveys strong information about possible parameter values it should overcome the priors, but still you are bringing out-of-data information into the model.

Idea similar to yours was described by Spiegelhalter (2004) proposed using "community of priors" (see also Fayers et al, 1997; 2000; Kass and Greenhouse, 1989), i.e. using different priors for different prior hypotheses and looking on how did they influence the estimates. For example, priors can be "optimistic", or "skeptical". The more informative prior, the more heavily you insist on certain parameter values range.

Some argue that such usage of priors is logically inconsistent (can you have multiple different beliefs about something?), but it is a rather philosophical problem. Also while such approach was described in several places, I had a hard time in finding examples of using it in practice.

---

Kass, R.E. and Greenhouse, J.B. (1989). A Bayesian perspective. Comment on “Investigating therapies of potentially great benefit: ECMO,” by J. H. Ware. Statistical Science, 4, 310-317.

Spiegelhalter, D. J. (2004). Incorporating Bayesian ideas into health-care evaluation. Statistical Science, 156-174.

Fayers, P.M., Ashby, D., & Parmar, M.K. (1997). Tutorial in biostatistics Bayesian data monitoring in clinical trials. Statistics in medicine, 16(12), 1413-1430.

Fayers, P.M., Cuschieri, A., Fielding, J., Craven, J., Uscinska, B., & Freedman, L.S. (2000). Sample size calculation for clinical trials: the impact of clinician beliefs. British journal of cancer, 82(1), 213.

---