# Objective vs. subjective Bayesian paradigms

What is the difference between objective and subjective Bayesian paradigms?
What objects or procedures do they define or interpret differently?
Is there any difference in their choice of methods?

• +1 My impression - which is not really an authoritative one - is that being an "objective" Bayesian tends to correlate with automatic recipes for finding priors like the Jeffreys prior, while "subjective" ones let their private beliefs dictate the choice of prior. It might also be the case - but I am even less sure about that - is that for the former group one is more likely to see discussion about the "true" parameter and whether some estimator is "close" to it. – Christoph Hanck Dec 13 '18 at 12:44
• @ChristophHanck, thank you for your input. I hope someone knowledgeable can put together a good summary/overview and post it as an answer, then we shall see how your points fit in. – Richard Hardy Dec 13 '18 at 13:23
• I am also a bit confused about whether here objective refers to empirical (learning prior from the data like ARD/SBL) or objective in the sense, that it gives identical results to frequentist results. – boomkin Jan 8 '19 at 8:18
• @boomkin, AFAIK there are schools of thought which identify themselves with "subjective Bayesians" and "objective Bayesians". I would like to know their main ideas and the differences between the former and the latter. Learning prior from the data is a feature that the two may or may not treat differently. – Richard Hardy Jan 8 '19 at 8:20
• +1 but, I find it better not to use the term 'Bayesian' as a nominalized adjective. The adjective 'Bayesian' classifies methods, not people. The placement of statisticians into a false dichotomy "as if one is adhering to just one and only one school of thought out of several schools of thought" may create difficulty in answering the question. It could be a trick question that can not be answered because it is 'incorrectly' stated in the first place (using an incorrect premise about existence of Bayesians). – Sextus Empiricus Jan 8 '19 at 8:48

This is a very confusing topic, mostly owing to the fact that there are two different ways in which the concept of “subjectivism” is commonly used in these discussions.$$^\dagger$$ It is made even more confusing by the fact that there is a class of “subjectivism” that is rooted in prior elicitation from experts, and this particular variation has to be fit into the philosophical categorisation of paradigms carefully. I will try to bring some clarity to this issue by setting out some different ways in which “subjectivism” is often interpreted, and then setting out broad areas of agreement among Bayesians, and areas where there is a divergence in philosophical and practical approaches. I expect there will be others who will disagree with my own views on this, but I hope this gives a good starting point for clear discussion.

Weak subjectivism: In this interpretation, the term "subjective" is used in its weaker sense, meaning merely that probability encapsulates the rational beliefs of a subject. (Some people, such as myself, prefer to use the term "epistemic" for this concept, since it does not actually require subjectivity in the stronger sense.)

Strong subjectivism: In this interpretation, the term "subjective" is used in its stronger sense, meaning that weak subjectivism holds, and furthermore, the subject's belief lacks any outside "objective" justification (i.e., two or more different subjects could all hold different beliefs, and none would be considered more or less wrong than the others).

In Bayesian analysis it is generally the case that the chosen sampling distribution has an objective justification rooted in some understanding of the sampling mechanism. However, there is rarely any available information pertaining to the parameter, other than in the sample data. This gives rise to three broad paradigms in Bayesian statistics, which correspond to different ways of determining the prior distribution.

Subjective Bayesian paradigm: This paradigm agrees with weak subjectivism, and further holds that any set of probabilistic beliefs is equally valid. So long as subjects use Bayesian updating for new data, it is legitimate to use any prior. Under this paradigm, the prior does not require any objective justification. In this paradigm there is a focus on disclosing the prior used, and then showing how this updates with new data. It is common in this method to include sensitivity analysis showing posterior beliefs under a range of prior beliefs.

Objective Bayesian paradigm: This paradigm also agrees with weak subjectivism, but prefers to additionally constrain prior beliefs (before inclusion of any data) so that they are objectively “non-informative” about the parameter. In this paradigm the prior is supposed to accurately reflect the lack of available information pertaining to the parameter, outside of the data. This usually entails adopting some theory for how to set the prior (e.g., Jeffrey’s, Jaynes, Bernardo reference priors, etc.) This paradigm holds that a set of probabilistic beliefs is to be preferred if it is based on a prior belief that is objectively determined and uninformative about the parameters in the problem of interest. It agrees that any set of probabilistic beliefs is consistent with the rationality criteria underlying Bayesian analysis, but views beliefs based on “bad” priors (too informative about the unknown parameter) as being worse than those based on “good” priors. In this paradigm the prior is chosen from some uninformative class, and then updated with new data to yield an objective answer to the problem.

Expert-prior Bayesian paradigm: This method is often viewed as part of the subjective paradigm, and is not usually identified separately, but I consider it a separate paradigm because it has elements of each view. This paradigm agrees with weak subjectivism, but like the objective Bayesian paradigm, it does not view all priors as equally valid. This paradigm treats present “priors” as posteriors from previous life experience, and so regards the prior beliefs of subject-matter experts as being superior to the prior beliefs of non-experts. It also recognises that those beliefs are probably based on data that has not been systematically recorded, and is not based on systematic use of probability theory, so it is not possible to decompose these existing expert priors into an original non-informative prior and the data that this expert observed. (And indeed, in the absence of systematic use of probability theory, the present expert “prior” is probably not even consistent with Bayesian updating.) In this paradigm the expert’s present “subjective” opinion is treated as being a valuable encapsulation of subject-matter knowledge, which is treated as a primitive prior. In this paradigm the analyst seeks to elicit the expert prior through some tests of prior belief, and then the prior is formulated as the best fit to that expert belief (taking care to ensure that the expert belief has not been polluted by knowledge of the present data). The “subjective” belief of the expert is thus treated as an “objective” encapsulation of subject-matter knowledge from previous data.

Differences in method: In terms of method, the objective Bayesian paradigm differs from the subjective paradigm insofar as the former constrains the allowable priors (either to a unique prior or a very small class of similar priors), whereas the latter does not constraint the allowable priors. In the objective Bayesian approach the prior is constrained by theories of representing a “non-informative” prior. The expert-prior paradigm takes a different approach and instead identifies one or more people that are experts, and elicits their prior beliefs.

Once we understand these different sense of the different paradigms in Bayesian statistics, we can set out some areas of broad agreement, and areas where there is disagreement. Actually, despite differences in method, there is more agreement on the underlying theories than is usually apprectiated.

Broad agreement on weak subjectivism: There exists a large literature in Bayesian statistics showing that the “axioms” of probability can be derived from preliminary desiderata relating to rational decision-making. This includes arguments pertaining to dynamic belief consistency (see e.g., Epstein and Le Breton 1993), arguments appealing to the Dutch book theorem (see e.g., Lehmann 1955, Hajek 2009). Bayesians of all these paradigms broadly agree that probability should be interpreted epistemically, as referring to the beliefs of a subject, constrained by the rationality constraints inherent in the axioms of probability. We agree that one should use the rules of probability to constrain one’s beliefs about uncertainty to be rational. This implies that beliefs about uncertainty require Bayesian updating in the face of new data, but it does not impose any further constraint (i.e., without more, it does not say that any prior is better than any other prior). All three of the above paradigms agree on this.

There is agreement that posteriors tend to converge with more data: There are a number of consistency theorems in Bayesian statistics which show that if you have two people with the same likelihood function, but different priors, then their posterior beliefs will converge as you get more and more data.$$^{\dagger \dagger}$$ This means that with a large amount of data, the prior does not matter very much. These are undeniable theorems of probability, and all three of the above paradigms agree on this. For this reason, it is generally recognised that with large amounts of data, any of the three paradigms is likely to give similar results. Consequently, the differences in the paradigms are most important when we only have a small amount of data.

There is broad agreement that there are roughly “objective” rules for priors that are available if you want to use them: There exists a large body of literature in Bayesian statistics showing how you can develop “non-informative” priors which are roughly determined by the sampling problem, and roughly encapsulate the absence of much knowledge about the parameter in question. I say “roughly” because there are several competing theories here that sometimes correspond but sometimes differ slightly (e.g., Jeffrey’s, Jaynes, reference priors, Walley classes of imprecise priors, etc.), and there are also some tricky paradoxes that can occur. The most difficult issue here is that it is difficult to make an “uninformative” prior for a continuous parameter that can be subjected to nonlinear transformations (since “uninformativity” should ideally be invariant to transformations). Again, these are theorems of probability, and all the paradigms agree with their content. Objective Bayesians tend to view this theory as being sufficiently good that it gives superior priors, whereas subjective Bayesians and expert-prior Bayesians tend to view the theory as being insufficient to establish superiority of these priors. In other words, there is broad agreement that these objective rules exist, and can be used, but there is disagreement over how good they are.

There is disagreement over the importance of having a single answer: Objective Bayesians are motivated by the preference that a statistical problem with fixed data and a fixed likelihood function should lead to a uniquely determined posterior belief (or at least a small number of allowable posterior beliefs that vary very little). This preference is generally part of a broader preference for having scientific procedures that yield a unique answer when applied to fixed sets of objective conditions. Contrarily, both subjective Bayesians and expert-prior Bayesians believe that this is not especially important, and they generally believe that this focus on a uniquely determined posterior is actually misleading.

There is broad agreement that the public are not well-acquainted with Bayesian posteriors: All paradigms agree that the general public are not well-acquainted with the basic mechanics of how Bayesian analysis transitions from a prior to a posterior. Objective Bayesians sometimes worry that giving more than one allowable answer for the posterior will be confusing to people. Subjective Bayesians worry that failing to give more than one allowable answer for the posterior is misleading to people.

$$^\dagger$$ It is worth noting that the confusion over "subjectivism" here stems from a particular instance of the general false dichotomy in epistemology between "subjectivism" and "intrinsicism" (see e.g., Piekoff). In attempts to interpret probability many users have made the error of believing that any rejection of aleatory theories of probability necessarily lead to interpretations that are "subjective" in the stronger sense specified here. To understand probability interpretations correctly, it is a good idea to understand the general problems with the subjectivism-intrinsicism dichotomy, and therefore recognise that objective epistemic interpretations exist.

$$^{\dagger \dagger}$$ There are some broad regularity conditions required for this result (e.g., both subjects have a prior with support including the true parameter value) but it applies very broadly.

• Thank you for such a voluminous and eloquent answer! So it looks like priors are the main roots of the disagreement. Is that it? Is there any disagreement/variations on, say, the definition/interpretation of probability? Or what it means for a parameter to be a random variable? I forgot to include this in the OP, but could you perhaps include a few names that can more or less clearly be assigned to a given paradigm? I know not all statisticians can be put in one or another box, but there must be some founding fathers or current devoted proponents in each camp. – Richard Hardy Jan 10 '19 at 6:57
• The root of the disagreement is definitely about the desirability/undesirability of objective inferences (i.e., unique of close to unique), and the scope of what constitutes a "good" prior. I don't think that is any broader disagreement on the interpretation of probability, or the meaning of a random variable. In my understanding, pretty-much all Bayesians agree on the epistemic interpretation. Since most of the basis for the paradigms is set in theorems, where there is no room for disagreement on validity, the only disagreements are about the value and desirability of different methods. – Ben Jan 10 '19 at 7:25
• Thank you very much! I am lucky I got you on my case. – Richard Hardy Jan 10 '19 at 7:37
• +1 Nice answer. One thing that seems to be missing is some mention of "empirical Bayes" methods where the prior is estimated from the data itself. I feel this should be a whole separate category along with your subjective priors, objective (aka uninformative) priors, and expert priors. – amoeba Jan 10 '19 at 11:57
• Ben, let me thank you again for your answer. I am keeping it unaccepted just in case someone in the future would offer a different perspective, but you certainly deserve the bounty you got. (I have another question with a bounty here that is about to expire, just in case you might be interested. I should of course have pinged you way earlier in this regard. Though there is normally a grace period of 24 hours after the expiration...) – Richard Hardy Jan 22 '19 at 10:33