Just How Independent are Events Anyways

Question

I want to describe a thought experiment to explore independent events, and whether they may actually be linked.

You are given a set of independent coin generators.
Coin generators are true random number generators that return 'Heads' or 'Tails'.
The probability that any of the independent coin generators would return heads is $0.5$. The set of all of these coin generators is $S$.
$\# S = n$.
You fire all the coin generators, and they all produce their results.
Before observing the coin generators, you split the coin generators into two groups $A$ and $B$.
The selection of each group is random. $\# A + \# B = n$
You get two pieces of paper and write out your expectation for the number of heads you expect for each group on each piece.
You then observe $A$.
Do you change your observation for $B$?
Should you?

The random variables of interest are ${\#}h(A)$ and ${\#}h(B)$.

A few things I feel need to be made explicit. As at the time you conducted the experiment (immediately after firing the coin generators and before splitting $S$), the number of heads in $S$ was fixed. Let's call the set of all heads in $S$ $H$.
For each $V \subset S$ the set of all heads in $V$ is $h(V)$.
 
The selection process is random, this means that $\frac{{\#}A}{{\#}S} : \frac{{\#}h(A)}{{\#}H}$ may be $\gt 1, \, \lt 1$ or $= 1$. However, for each $A$ and $S$, only a few selections ($n\choose {\#}A$) produce $\frac{{\#}A}{{\#}S} : \frac{{\#}h(A)}{{\#}H} = 1$.

$A = B'$ and vice versa.
 
Let's call the sampling from $S$ such that $\frac{{\#}A}{{\#}S} = \frac{{\#}h(A)}{{\#}H}$ the equal sampling $(T_E)$.
Let's call the sampling from $S$ such that $\frac{{\#}A}{{\#}S} \gt \frac{{\#}h(A)}{{\#}H}$ the $A$-biased sampling $(T_A)$.
Let's call the sampling from $S$ such that $\frac{{\#}A}{{\#}S} \lt \frac{{\#}h(A)}{{\#}H}$ the $B$-biased sampling $(T_B)$.
 
$Pr(T_E) = n{\choose}{\#}A$ $$Pr(T_A) = Pr(T_B) = \frac{2^n - {{n}\choose{\#}A}}{2}$$

I feel the above should be kept in mind when we consider whether $h(A)$ affects $E\left(h(B)\right)$. For one, observing a higher than expected amount of heads in $A$ may raise the probability of the $A$-biased sampling.  
I think that depending on what I observed in $A$, that I may revise my beliefs about $B$. Assuming $n = 8$, ${\#}A = 6$, and $h(A) = 6$, I will update (an increase) my posterior probability on the selection chosen favouring $A$ over $B$ in distribution of heads. This may in turn lead me to lower my estimation of the heads contained in $B$.

I think my stance of possibly shifting my beliefs about $B$ based on my observation of $A$ is legitimate, as the selection process is random and may lead to an unfair distribution of heads.

Should we shift our beliefs after observing $A$ (and more importantly, why so)?
 
 
 

NOTE:

I think it is important to draw a distinction between this experiment and the gambler's fallacy. We do not first generate $A$ and then subsequently generate $B$ (in which case $h(A)$ would be independent of $h(B)$), we generate $S$. We do not observe the number of heads or tails in $S$. The coin generators may be fired in any order (temporal or otherwise), but we do not observe $S$ until after all $x \in S$ have finished generating.
 
We then randomly sample $S$ into $A$ and $B(A')$. After sampling $S$, we estimate the number of heads in $A$ and $B$ $\left(E\left(h(A)\right) \text{and} \, E\left(h(B)\right) \text{respectively}\right)$. After our estimation, we observe $A$: should we update $E\left(h(A)\right)$ in light of $h(A)$? I don't think it's as clear cut as people seem to be making it out to be.

Answer 1

Since you know, a priori, that the outcome of each generator is iid Bernoulli(0.5), then there is no basis for you to update your estimate of $E[h(A)]$ or $E[h(B)]$. The reason is that you didn't say you know the total number of heads (#$H$), so observing $A$ doesn't constrain what $B$ could be.

The reason is that the outcomes from each generator are exchangeable, and so knowledge of one partition doesn't help you in understanding the other partition.

The issue of when the generators were fired is irrelevant, the basic theory is that, by the iid assumption, $P(S_1,S_2|S_3,S_4,...,S_n) = P(S_1,S_2)$, so your expected value calculations shouldn't change.

So what could be conditional?

Say you know $h(A) + h(B) = N$, then, even if the generators are independent, you know $h(A)$ you know that $E[h(B)] = N - h(A)$.

Another case is if we drop the constraint $h(A) + h(B) = N$, we still have $E[h(S)|h(A)] = h(A) + \frac{\#B}{2}$

That's one way you would get conditional expectation to differ from unconditional even in the case of independence...but it requires altering the sample space to reflect new knowledge.

Your derivations of $A-$biased, $B$-biased are only relevant if you know $N$...if you don't then its not relevant.

## R Code Example ##

Let's simulate this process to show what it looks like:

h_A <- c()
h_B <- c()
n_A <- c()
n_B <- c()

for(i in 1:1000){
  S <- rbinom(10,1,0.5) #fire generators
  idx_A <- sample(1:length(S),round(runif(1,0,length(S))),replace = FALSE) #select A
  A <- S[idx_A] #assign values for A
  B <- S[setdiff((1:10),idx_A)] #assign rest to B
  h_A <- append(h_A,sum(A))
  n_A <- append(n_A,length(A))
  n_B <- append(n_B,length(B))
  h_B <- append(h_B,sum(B))
}

plot(h_A,h_B)

#look at conditional means
A_values <- unique(h_A)
uncond_mean <- c()
cond_mean <- c()

for(i in 1:length(A_values)){
  val <- A_values[i]
  subset_A <- h_A==val #where did we see h_A == val
  n <- n_B[subset_A] #size of B partition
  selected_B <- h_B[subset_A] #what was the number of heads in B?
  uncond_mean <- append(uncond_mean,mean(n/2)) #just find mean of 10-N independent tosses
  cond_mean <- append(cond_mean,mean(selected_B)) #actually calculate mean of B
}

plot(uncond_mean,cond_mean)

lm(cond_mean ~ uncond_mean)

You get as output:

Now, look at conditional vs unconditional means:

Call:
lm(formula = cond_mean ~ uncond_mean)

Coefficients:
(Intercept)  uncond_mean  
    0.01645      0.98479

So, the actual heads tracks the unconditional mean