# Entropy of independent draws of balls from a bin

Problem Setting

Consider a bin containing $B$ balls, each ball is red or blue with equal probability ($0.5$). Let $R$ denote the number of red balls. It is easy to see that $R \sim Binomial(B,0.5)$

In each round, a ball is drawn with replacement. The color of the ball drawn in the $i^{th}$ round is denoted by random variable $X_i$.

Question

What is the conditional entropy of $Y_k = R\big|X_1,X_2,\cdots,X_k$?

$$H(Y_k) = H\big(R\big|X_1,X_2,\cdots,X_k\big)$$

In words, $Y_k$ is a random variable denoting the number of red balls after seeing the color of balls drawn after $k$ independent draws.

• When you say $Bern$ do you actually mean binomial? – Glen_b Jul 21 '16 at 4:47
• Write down the probability of observing $X_i$ given $R$ and use Bayes' rule. – Neil G Jul 21 '16 at 5:18
• @Neil: It is a good idea. But the expression obtained after Baye's rule is very messy. – Vivek Bagaria Jul 21 '16 at 18:32