I have used Bayesian reasoning in my research work and it has been extremely useful. The book I have read is E.T. Jayne's Probability theory. The idea is to formulate propositions and then probability theory tells how to assign numbers (viz. probability) to those propositions, conditional on one's information and beginning from some prior probabilities.

A proposition is something that is decidedly either true or false, irrespective of any observer. Therefore for a given coin "Probability of Heads is $p$" is not a proposition, because in Bayesian view probability depends on the observer (his/her information) and is not an objective property of the coin (like its mass or temperature).

Suppose I toss the coin once and get Heads. I ask "What's the probability of Heads in the next toss?" Consider the propositions: $$H_k\equiv\textrm{Heads in $k$-th toss}\\ T_k\equiv\textrm{Tails in $k$-th toss}$$

Then my question is: $P(H_2|H_1)=?$

I assume uniform prior probabilities: $P(H_k)=P(T_k)=1/2$ for any $k$. The result of first toss must change the probability of Heads in the second toss (I'm not assuming that the coin is fair; if say 100 tosses were to turn up Heads then I would suspect the coin to be biased in favour of Heads and probability theory must indicate the same to me).

Bayes rule gives: $$P(H_2|H_1)=\frac{P(H_1|H_2)P(H_2)}{P(H_1)}=P(H_1|H_2)$$

This gets me nowhere. How do I get a number and thus update the probability of Heads with each toss?

In this post and some articles I read on the net, this issue is resolved by taking "Probability of Heads is $p$" as a proposition and then seeking its probability (which amounts to seeking the probability of a probability). This does give an answer. My only problem (which I believe is a major problem) is that the aforementioned statement is not a proposition and so asking for its probability is nonsense. What's a way out of this conundrum?

  • $\begingroup$ Not enough framework provided here. // If you assume coin tosses are independent, then $P(H_2|H_1) = P(H_2).$ If you have some mechanism by which the coin 'remembers' its past behavior, then you need to describe that. // If you assume coin has same probability of heads on each toss then $P(H_1) = P(H_2).$ // In a true Bayesian context, you would start with a prior distribution on $\theta = P(H).$ $\endgroup$
    – BruceET
    Sep 17 '18 at 7:35
  • $\begingroup$ @BruceET I don't have a reason to assume independence. As I said in the post, if suppose 100 successive tosses came up Heads, I would suspect that the coin is biased in favour of Heads, despite not knowing the mechanism that causes bias (in fact Bayesian inference doesn't require one to know the real world mechanisms that may affect an experiment and that is precisely its strength); the probability of Heads in 101-th toss, conditional on the fact that previous 100 tosses were Heads, ought to be more than 0.5. Bayes updating should result in this reasonable conclusion. $\endgroup$
    – Deep
    Sep 18 '18 at 5:36
  • $\begingroup$ (contd.) If you take it to an extreme and say 1 billion tosses came up Heads, then it is reasonable to conclude that the coin is heavily biased in favour of Heads (and again you don't need to know the underlying mechanism that causes the bias), and therefore the probability of Heads coming up in the next toss must be close to 1. Also in the post I have assumed the prior to be 1/2 for Heads. $\endgroup$
    – Deep
    Sep 18 '18 at 5:38
  • $\begingroup$ So you have the ultimate in informative priors, degenerate at $1/2.$ Interesting approach. Certainly simplifies things. $\endgroup$
    – BruceET
    Sep 18 '18 at 7:12
  • $\begingroup$ @BruceET I am not sure I understand your comment. Should I have assumed a different prior? The problem is that I don't know how to do Bayesian updating in my case (so that I get a number for probability of Heads in the next toss) and this problem persists no matter which prior I choose. $\endgroup$
    – Deep
    Sep 18 '18 at 8:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.