1
$\begingroup$

Problem Setting

Consider a bin containing $B$ balls, each ball is red or blue with equal probability ($0.5$). Let $R$ denote the number of red balls. It is easy to see that $R \sim Binomial(B,0.5)$

In each round, a ball is drawn with replacement. The color of the ball drawn in the $i^{th}$ round is denoted by random variable $X_i$.

Question

What is the conditional entropy of $Y_k = R\big|X_1,X_2,\cdots,X_k$?

$$ H(Y_k) = H\big(R\big|X_1,X_2,\cdots,X_k\big) $$

In words, $Y_k$ is a random variable denoting the number of red balls after seeing the color of balls drawn after $k$ independent draws.

$\endgroup$
  • 2
    $\begingroup$ When you say $Bern$ do you actually mean binomial? $\endgroup$ – Glen_b Jul 21 '16 at 4:47
  • $\begingroup$ Write down the probability of observing $X_i$ given $R$ and use Bayes' rule. $\endgroup$ – Neil G Jul 21 '16 at 5:18
  • $\begingroup$ @Neil: It is a good idea. But the expression obtained after Baye's rule is very messy. $\endgroup$ – Vivek Bagaria Jul 21 '16 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.