# Do Bayesians accept Kolmogorov's axioms?

Usually probability theory is taught with Kolgomorov's axioms. Do Bayesians also accept Kolmogorov's axioms?

• Bayesian theory follows from the standard axioms of probability, hence from Kolmogorov axioms. Commented Nov 30, 2014 at 21:21
• @Xi'an: That subjective degrees of belief can be represented by probability isn't so obvious - hence the question, & Ramsey & de Finetti's work. Commented Nov 30, 2014 at 21:49
• That's why I am an "objective" Bayesian and start with prior distributions defined according to the standards of probability theory... Commented Nov 30, 2014 at 21:53
• I believe that Cox-Jaynes interpretation of probability provides a rigorous foundation for Bayesian probability. (see my answer). However, it would be nice to have Xi'an's opinion on that. Commented Dec 1, 2014 at 11:27
• Here is a very recent arXiv paper addressing the issue. Commented Dec 3, 2014 at 9:17

In my opinion, Cox-Jaynes interpretation of probability provides a rigorous foundation for Bayesian probability:

• Cox, Richard T. "Probability, frequency and reasonable expectation." American Journal of Physics 14.1 (1946): 1–13.
• Jaynes, Edwin T. Probability theory: the logic of science. Cambridge University Press, 2003.
• Beck, James L. "Bayesian system identification based on probability logic." Structural Control and Health Monitoring 17.7 (2010): 825–847.

The axioms of probability logic derived by Cox are:

1. (P1): $$\Pr[b|a]\ge0$$ (by convention)
2. (P2): $$\Pr[\overline{b}|a]=1-\Pr[b|a]$$ (negation function)
3. (P3): $$\Pr[b\cap c|a]=\Pr[c|b\cap a]\Pr[b|a]$$ (conjunction function)

Axioms P1-P3 imply the following [Beck, 2010]:

1. (P4): a) $$\Pr[b|b\cap c] = 1$$; b) $$\Pr[\overline{b}|b\cap c] = 0$$; c) $$\Pr[b|c]\in[0,1]$$
2. (P5): a) $$\Pr[a|c \cap (a \Rightarrow b)]\le\Pr[b|c\cap(a \Rightarrow b)]$$, b) $$\Pr[a|c\cap(a \Leftrightarrow b)] = \Pr[b|c\cap(a \Leftrightarrow b)]$$, where $$a \Rightarrow b$$ means that $$a$$ is contained in $$c$$, and $$a \Leftrightarrow b$$ means that $$a$$ is equivalent to $$b$$.
3. (P6): $$\Pr[a \cup b|c] = \Pr[a|c]+\Pr[b|c]-\Pr[a\cap b|c]$$
4. (P7): Assuming that proposition $$c$$ states that one and only one of propositions $$b_1,\ldots,b_N$$ is true, then:
• a) Marginalization Theorem: $$\Pr[a|c]=\sum_{n=1}^N P[a \cap b_n|c]$$
• b) Total Probability Theorem: $$\Pr[a|c] = \sum_{n=1}^N \Pr[a|b_n\cap c]\Pr[b_n|c]$$
• c) Bayes' Theorem: For $$k=1,\ldots,N$$: $$\Pr[b_k|a\cap c] = \frac{\Pr[a|b_k\cap c]\Pr[b_k|c]}{\sum_{n=1}^N \Pr[a|b_n\cap c]\Pr[b_n|c]}$$

They imply Kolmogorov's statement of logic, which can be viewed as a special case.

In my interpretation of a Bayesian viewpoint, everything is always (implicitly) conditioned on our believes and on our knowledge.

The following comparison is taken from Beck [2010]:

# The Bayesian point of view

Probability is a measure of plausibility of a statement based on specified information.

1. Probability distributions represent states of plausible knowledge about systems and phenomena, not inherent properties of them.
2. Probability of a model is a measure of its plausibility relative to other models in a set.
3. Pragmatically quantifies uncertainty due to missing information without any claim that this is due to nature's inherent randomness.

# The Frequentist point of view

Probability is the relative frequency of occurrence of an inherently random event in the long run.

1. Probability distributions are inherent properties of random phenomena.
2. Limited scope, e.g., no meaning for the probability of a model.
3. Inherent randomness is assumed, but cannot be proven.

# How to derive Kolmogorov's axioms from the axioms above

In the following, section 2.2 of [Beck, 2010] is summarized:

In the following we use: probability measure $$\Pr(A)$$ on subset $$A$$ of a finite set $$X$$:

1. [K1] $$\Pr(A)\ge 0, \forall A \subset X$$
2. [K2]: $$\Pr(X) = 1$$
3. [K3]: $$\Pr(A\cup B)=\Pr(A)+\Pr(B), \forall A,B \subset X$$ if $$A$$ and $$B$$ are disjoint.

In order to derive (K1–K3) from the axioms of probability theory, [Beck, 2010] introduced propositon $$\pi$$ that states $$x\in X$$ and specifies the probability model for $$x$$. [Beck, 2010] furthermore introduces $$\Pr(A) = \Pr[x\in A|\pi]$$.

• P1 implies K1 with $$b=\{x\in A\}$$ and $$c=\pi$$
• K2 follows from $$\Pr[x\in X|\pi]=1$$; P4(a), and $$\pi$$ states that $$x\in X$$.
• K3 can be derived from P6: $$A$$ and $$B$$ are disjoint means that $$x\in A$$ and $$x\in B$$ are mutually exclusive. Therefore, K3: $$\Pr(x\in A\cup B|\pi)=\Pr(x\in A|\pi)+\Pr(x\in B|\pi)$$
• From your K3 you can get to $\Pr(\cup_{i=1}^n A_i)=\sum_{i=1}^n\Pr(A_i)$ (finite additivity) but not to Kolmogorov's 3rd axiom, $\Pr(\cup_{i=1}^\infty A_i)=\sum_{i=1}^\infty\Pr(A_i)$ (countable additivity) when the $A$'s are elements of a $\sigma$-field, & not simply subsets of a finite set. Commented Dec 2, 2014 at 10:04
• @Scortchi K.R.Koch in his introduction to Bayesian Statistics cites Bernardo and Smith (1994), Bayesian Theory, p. 105, as a source that shows how to address countable infinity. I have not checked it, but as a reference it may as well be given here.
– gwr
Commented Aug 23, 2016 at 22:22

After the development of Probability Theory it was necessary to show that looser concepts answering to the name of "probability" measured up to the rigorously defined concept they had inspired. "Subjective" Bayesian probabilities were considered by Ramsey and de Finetti, who independently showed that a quantification of degree of belief subject to the constraints of comparability & coherence (your beliefs are coherent if no-one can make a Dutch book against you) has to be a probability.

Differences between axiomatizations are largely a matter of taste concerning what should be defined & what derived. But countable additivity is one of Kolmogorov's that isn't derivable from Cox's or Finetti's, & has been controversial. Some Bayesians (e.g. de Finetti & Savage) stop at finite additivity & so don't accept all of Kolmogorov's axioms. They can put uniform probability distributions over infinite intervals without impropriety. Others follow Villegas in also assuming monotone continuity, & get countable additivity from that.

Ramsey (1926), "Truth and probability", in Ramsey (1931), The Foundations of Mathematics and other Logical Essays

de Finetti (1931), "Sul significato soggettivo della probabilità", Fundamenta Mathematicæ, 17, pp 298 – 329

Villegas (1964), "On qualitative probability $$\sigma$$-algebras", Ann. Math. Statist., 35, 4.

• Why should my answer only deal with 'objective Bayesian' probabilities? The seminal work of Cox (1946) explicitly addresses the issue of subjectivity! It is a very interesting - and easy to read paper. I do not think that it makes sense to distinguish between 'subjective' and 'objective' Bayesian probabilities: Everything is always implicitly conditioned to the person performing the analysis -> and in this regard 'subjective'. Commented Dec 1, 2014 at 13:58
• concerning the derivation of the axioms stated Kolmogorov's from Cox's: I am satisfied by the way it is done in section 2.2 of Beck, James L. "Bayesian system identification based on probability logic." Structural Control and Health Monitoring 17.7 (2010): 825-847. Commented Dec 1, 2014 at 14:20
• @Summit: (1) You're right; it's rather that Ramsey & de Finetti's dispositional view of probability puts them squarely in the "subjective" camp, whereas Cox's is more generally applicable. (2) Are you saying countable additivity can be deduced from Cox's postulates? Commented Dec 1, 2014 at 17:03
• I extended my answer, and look forward to your comments. Commented Dec 1, 2014 at 18:32
• @Summit: Thanks - I hope to find time to make mine even half as thorough. I've pointed out the gap between where you can get to from Cox's theorem & the "full" Kolmogorov axioms & think it's especially germane to the question (though I'd forgotten about it entirely when I first answered). Jaynes had some interesting things to say about this BTW. Commented Dec 3, 2014 at 18:20