Are two coin flips conditionally independent if we know that the coin is biased towards heads?

Question

Suppose Alice (A) and Bob (B) each flip the same, potentially-biased coin. Then, P(A=H) < P(A=H | B=H), because Bob's flip increases our suspicion that the coin is biased towards heads.

Now instead suppose that we already know that the coin is biased towards heads. Now is it true that P(A=H) < P(A=H | B=H)? It seems clear to me that the answer is still yes, because we don't know how much it is biased, and B=H still suggests that it is more biased than if B=T.

To be more precise, regardless of the prior $P(\theta)$ we pick (where $\theta$ is the odds of heads), the posterior $P(\theta | H)$ will still be shifted right. It doesn't matter whether the prior is uniform on [0, 1] (example 1) or (0.5, 1] (example 2).

But this post at The Queen Mary University of London (and this answer at Math SE) seems to suggest otherwise:

Now suppose both Martin and Norman toss the same coin. Again let A represent the variable "Norman's toss outcome", and B represent the variable "Martin's toss outcome". Assume also that there is a possibility that the coin in biased towards heads but we do not know this for certain. In this case A and B are not independent. For example, observing that B is Heads causes us to increase our belief in A being Heads (in other words P(a|b)>P(b) in the case when a=Heads and b=Heads).

In Example 2 the variables A and B are both dependent on a separate variable C, "the coin is biased towards Heads" (which has the values True or False). Although A and B are not independent, it turns out that once we know for certain the value of C then any evidence about B cannot change our belief about A. Specifically:

P(A|C) = P(A|B,C)

In such case we say that A and B are conditionally independent given C.

What am I missing?

Edit: This is increasingly striking me as a frequentist vs Bayesian issue. In scenario 1, we're behaving like a Bayesian (incorporating our knowledge about the parameter into our estimate) and scenario 2 like a frequentist (treating it as a fixed value and therefore not updating it based on data).

I think what's confusing me is that there seems to be no principled reason to switch between the two paradigms, other than a subtlety of wording ("there is a possibility that the coin in biased" in the first scenario). A strong frequentist should apply her paradigm to both cases, and likewise a Bayesian. Would you agree?

Answer 1

The quoted section is implicitly assuming that the event $C = \{ \theta > 0.5 \}$ is sufficient to fully describe the parameter, and so it attains conditional independence of the observable coin flips (e.g., there may be an assumption that there is only one allowable value of $\theta$ in the biased range). Contrarily, your own analysis is saying that even if $C$ is true, there is still uncertainty parameter value, so the coin flips still give information about the underlying parameter $\theta$, and so they remain dependent. Your analysis here is more realistic, and I agree with your assertion that there would still be dependence even once you condition on $C$.

This issue has been discussed in detail in O'Neill (2009), which looks at conditional independence and marginal dependence in exchangeable sequences of random variables. You can also find some associated theorems for statistical dependence in coin-flipping in a series of papers on binomial prediction (see O'Neill and Puza 2005; O'Neill 2012; O'Neill 2015). These latter papers discuss the "gambler's fallacy", and show that ---under broad conditions--- one obtains a predictive advantage by betting on whichever outcome of the coin-flip has come up the most in the observed data (to take advantage on information about possible bias).