29
$\begingroup$

In Jaynes' book "Probability Theory: The Logic of Science", Jaynes has a chapter (Ch 18) entitled "The $A_p$ distribution and rule of succession" in which he introduces the idea of $A_p$ distributions, which this passage helps illustrate:

[...] To see this, imagine the effect of getting new information. Suppose we tossed the coin five times and it comes up tails every time. You ask me what’s my probability for heads on the next throw; I’ll still say 1/2. But if you tell me one more fact about Mars, I’m ready to change my probability assignment completely [that there was once life on Mars]. There is something which makes my state of belief very stable in the case of the penny, but very unstable in the case of Mars

This might seem to be a fatal objection to probability theory as logic. Perhaps we need to associate with a proposition not just a single number representing plausibility, but two numbers: one representing the plausibility, and the other how stable it is in the face of new evidence. And so, a kind of two-valued theory would be needed. [...]

He goes on to introduce a new proposition $A_p$ such that $$P(A|A_pE) ≡ p$$

"where $E$ is any additional evidence. If we had to render $A_p$ as a verbal statement, it would come out something like this: $A_p$ $≡$ regardless of anything else you may have been told, the probability of $A$ is $p$."

I'm trying to see the distinction between the two-number idea ("plausibility, and the other how stable it is in the face of new evidence") with just using the Beta distribution which satisfies those criteria.

Fig 18.2 is very similar to using $\alpha=\beta=100$ (say), whereas for Mars it could be Beta(1/2,1/2) and the state of belief is "very unstable"

enter image description here

The original $A_p$ proposition, above, could be Beta($\alpha,\beta$) for very large $\alpha,\beta$ such that $\alpha$/($\alpha+\beta)=p$. Then no amount of evidence would change the distribution of $p$ and $P(A|A_pE) ≡ p$

Beta distribution is discussed throughout the book, so am I missing something that the distinction here is subtle and warranting a new theory ($A_p$ distribution)? He does mention in the very next paragraph "It seems almost as if we are talking about the ‘probability of a probability’."

Improve this question
$\endgroup$
6
  • 3
    $\begingroup$ I am not sure, but maybe Dempster-Shafer theory is something to ponder in this line of thought? On the other hand, models may be dynamic and hierarchical in Bayesian statistics -- thus woudln't it be possible to model probability of stability within the regular Bayesian framework? $\endgroup$ – gwr Nov 13 '15 at 11:38
  • 4
    $\begingroup$ We, the CV readership, don't have access to "Fig. 18.2." If it's important enough, would it be possible to provide a link? One thing worth noting is that α=β for both the coin toss and Mars. If α/(α+β)=p then it would appear that α is your statement of confidence, based on the Beta distribution. I was surprised that Jaynes treatment of plausibility didn't discuss C.S. Peirce's work. Peirce was a giant in 19th and early 20th c American philosophy who made some very apposite comments regarding the statistical foundations of plausibility plato.stanford.edu/entries/peirce/#prob $\endgroup$ – Mike Hunter Nov 13 '15 at 12:15
  • 7
    $\begingroup$ (Entirely orthogonal comment: Surnames like Jaynes are awkward to handle even for people with English as their first language. Jaynes' and Jaynes's would both have defenders as possessives, but they are the only possible possessives. It's easy to slip into writing Jayne's (quite wrong in this case) if the name is misunderstood.) $\endgroup$ – Nick Cox Nov 13 '15 at 14:13
  • $\begingroup$ It seems to me that, as you suspect, Jaynes's idea is basically just the Bayesian view of probability. Edwin Jaynes died in 1998, so we can't ask him, and there's not much evidence he meant something meaningfully different, so it seems that that's all that can be said on the matter. $\endgroup$ – Kodiologist Jul 20 '17 at 20:35
  • $\begingroup$ I think this sits well with the Bayesian approach. You could definitely model using the beta distributions, with a highly informative prior based on previous evidence and intuition (the penny) and a highly uninformative prior (life on Mars). But I think the more interesting thing is to look at the space of potential evidence in each case. There is evidence that would completely change our theory about the coin - if it came up feet, say, or pineapples, then we would have to rethink our understanding just as much. $\endgroup$ – David DAemon Allen Jan 3 '20 at 11:44
1
$\begingroup$

The main point of $A_p$ theory is to describe how cumulative knowledge changes the probability distributions of new observations, given a model.

Starting from a full ignorance model, consistent observations increase the strength of a belief (narrow variance of $(A_p|E)$), while ignorance or inconsistency generate larger instability of the belief (greater variance of $(A_p|E)$.

By introducing $A_p$, Jayes:

  1. shows that a "brain" only needs to store $(A_p|E)$ to make future predictions
  2. gives a model of inductive reasoning
  3. underlines the importance of Laplace's rule of succession

Jaynes argues that the $A_p$ densities are the core idea to describe how human brain summarizes previous experience into models (inner robot). The inner robot continuously encodes experience updating $A_p$ and combining it into complex forms can be the basis of creativity (Chapter 18.4).

In practice, it is like you said. The difference between the $A_p$ for the coin and Mars examples are captured by the parameters of the $\text{Beta}$ distributions, but the concept is deeper than that.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.