Deriving eq. 3.56 in Jaynes' Probability Theory using expectations

Question

Assuming $R_{\text{later}}=R_{k+1}+R_{k+2}+\ldots+R_n$ and that B (background information), in summary is: There are $N$ balls in an urn, $M$ of them being red and we draw $n$. Jaynes derives the probability for drawing a red ball in the $k$th draw as: $$P(R_k|R_{\text{later}}B)=\frac{M}{N-n+k}\times\frac{\binom{N-1}{n-k}-\binom{N-M}{n-k}}{\binom{N}{n-k}-\binom{N-M}{n-k}}$$ I have no problem with this proof when he first derives it, but then (after defining the expectation) he says it can be proved using the expectation of a fraction. I am having a problem trying to do that. Below you can see how I have attempted to derive this equation from the expectation of a fraction.

Since we know that $n-k$ balls will definitely be drawn, the effective number of balls left is: $N-(n-k)$, so we can write: $$P(R_k|R_{\text{later}}B)=\frac{\langle{M}\rangle}{N-n+k}$$ To find $\langle{M}\rangle$, using the basic definition we can write: $$\langle{M}\rangle=\sum_{m=1}^{N}\sum_{\text{over permutations}}mP(R_{k+1}\dots R_{k+m}W_{k+m+1}\dots W_{n}|R_{\text{later}}B)$$ To summarize I will take: $RW_m\equiv R_{k+1}\dots R_{k+m}W_{k+m+1}\dots W_{n}$. No matter how $R$ and $W$ are ordered, for a particular $m$ (fixed number of $R$ and $W$) the probability doesn't change, so we can write $\langle{M}\rangle$ as: $$\langle{M}\rangle=\sum_{m=1}^{N}m\binom{n-k}{m}P(RW_m|R_{\text{later}}B)$$ Now, $$P(RW_m|R_{\text{later}}B)=\frac{P(RW_mR_{\text{later}}|B)}{P(R_{\text{later}}|B)}=\frac{P(RW_m|B)}{P(R_{\text{later}}|B)}$$

Previously he has derived $P(R_{\text{later}}|B)$ as: $$P(R_{\text{later}}|B)=1-\binom{N-M}{n-k}\binom{N}{N-K}^{-1}$$

So the only element left is $P(RW_m|B)$. I am trying to find this using eq3.15 of the book:$$P(R_1\dots R_rW_{r+1}\dots W_n|B)=\frac{M!(N-M)!(N-n)!}{(M-r)!(N-M-n+r)!N!}$$But I am having problems trying to correctly use this formula. I am using these replacements: $r\rightarrow{m}$, $n\rightarrow{n-k}$ and $N$ is unchanged. But I don't understand how $M$ should be replaced in this equation to fit the purpose of this proof.

I would be grateful if any one could guide me in the correct solution of this proof.

Answer 1

I was happy to see the question already asked, but sad that there is still no answer.

I was also trying to do the same derivation and proceeded in the same direction as the OP (is that the right term? Sorry this is my first time answering) and got a little stuck. After finding this question, and in the process of typing a reply, I actually managed to solve it completely :). Here is the proof.

The approach is very similar to the OP with one difference of the expectation $\langle M\rangle$ is defined. When exactly one red ball occurs in the last $n-k$ draws, then the number of remaining red balls is $M-1$. Now extending this we have number of remaining red balls, $M-1,M-2,\dots,M-(n-k)$ when the number of red balls drawn are $1,2,\dots,n-k$. Now using your definition of $P(RW_i|R_{\text{later}}B)=P(R_{k+1}R_{k+2}\dots R_{k+i} W_{k+i+1}\dots W_{n}|R_{\text{later}}B)$, the expectation is

$$ \langle M\rangle = \sum_{i=1}^{n-k} (M-k){n-k \choose i}P(RW_i|R_{\text{later}}B) $$

Proceeding similarly to OP, we have $P(R_{\text{later}}|B)$ already derived and all that is left is to compute $P(RW_i|B)$. Now, using Eq (3.15) from the book, where we replace $n\rightarrow n-k, r\rightarrow i$, we get $$ P(RW_i|B) \equiv P(R_1\dots R_i W_{i+1}\dots W_{n-k}) = \frac{M!(N-M)!(N-(n-k))!}{(M-i)!(N-M-(n-k-i))!N!} $$

Substituting all the terms in the expectation and expanding binomial coefficients \begin{align} \langle M\rangle &= \sum_{i=1}^{n-k} (M-k){n-k \choose i}P(RW_i|R_{\text{later}}B)\\ &= \frac{{N \choose n-k}}{{N \choose n-k}-{N-M \choose n-k}}\sum_{i=1}^{n-k}(M-i){n-k \choose i} \frac{M!(N-M)!(N-(n-k))!}{(M-i)!(N-M-(n-k-i))!N!}\\ &= \frac{{N \choose n-k}}{{N \choose n-k}-{N-M \choose n-k}}\sum_{i=1}^{n-k}(M-i)\frac{(n-k)!}{(n-k-i)!i!} \frac{M!(N-M)!(N-(n-k))!}{(M-i)!(N-M-(n-k-i))!N!}\\ \end{align}

Now grouping the terms to form three binomial coefficients, \begin{align} =&\frac{{N \choose n-k}}{{N \choose n-k}-{N-M \choose n-k}}\sum_{i=1}^{n-k}(M-i)&\left[ \frac{(N-M)!}{(N-M-(n-k-i)!(n-k-i)!)} \right]\\ & &\left[ \frac{(N-(n-k)!(n-k)!)}{N!}\right]\\ & &\left[\frac{M!}{(M-i)!i!}\right]\\ \end{align} \begin{align} &= \frac{{N \choose n-k}}{{N \choose n-k}-{N-M \choose n-k}}\sum_{i=1}^{n-k}(M-i){N-M\choose n-k-i}{N\choose n-k}^{-1}{M \choose i}\\ &= \frac{1}{{N \choose n-k}-{N-M \choose n-k}}\sum_{i=1}^{n-k}(M-i){M \choose i}{N-M\choose n-k-i}\\ \end{align}

At this stage the product of the two binomial coefficients inside the summation is similar to identity 3.79 from the book (which I found out has the name Vandermonde's identity) stated as $$ {N1+N2 \choose r_a} = \sum_{r1=0}^{r_a}{N_1 \choose r_1}{N_2 \choose r_a-r_1} $$

Split the summation into two terms as follows: \begin{equation} \label{eq1} \langle M\rangle = \frac{1}{{N \choose n-k}-{N-M \choose n-k}}\left[M\sum_{i=1}^{n-k}{M \choose i}{N-M\choose n-k-i} -\sum_{i=1}^{n-k}i{M \choose i}{N-M\choose n-k-i}\right] \end{equation} Taking the first term and applying the identity: \begin{align} M\sum_{i=1}^{n-k}{M \choose i}{N-M\choose n-k-i} &= M\left\{ \sum_{i=0}^{n-k}{M \choose i}{N-M\choose n-k-i} - {M \choose 0}{N-M\choose n-k}\right\} \\ &= M\left\{ {N \choose n-k} - {N-M \choose n-k} \right\} \end{align} Taking the second term and applying the identity: \begin{align} \sum_{i=1}^{n-k}i{M \choose i}{N-M\choose n-k-i} &= \sum_{i=1}^{n-k}i\frac{M}{i}{M-1 \choose i-1}{N-M\choose n-k-i}\\ &= M\sum_{i=1}^{n-k}{M-1 \choose i-1}{N-M\choose n-k-i}\\ &= M\sum_{i=0}^{n-k-1}{M-1 \choose i}{N-M\choose n-k-i-1}\\ &= M{N-1\choose n-k-1}\\ \end{align}

Putting these two terms back in \ref{eq1} and taking the $M$ common, \begin{align} \langle M\rangle &= \frac{M}{{N \choose n-k}-{N-M \choose n-k}}\left[{N \choose n-k} - {N-M \choose n-k} - {N-1\choose n-k-1} \right]\\ &= \frac{M}{{N \choose n-k}-{N-M \choose n-k}}\left[{N \choose n-k} - {N-M \choose n-k} - {N-1\choose n-k-1} \right]\\ &= \frac{M}{{N \choose n-k} -{N-M \choose n-k}}\left[{N-1 \choose n-k}+ {N-1\choose n-k-1} - {N-M \choose n-k} - {N-1\choose n-k-1} \right]\\ &= \frac{M}{{N \choose n-k} -{N-M \choose n-k}}\left[{N-1 \choose n-k} - {N-M \choose n-k} \right]\\ \end{align} Where, the third equality comes from using Pascal's identity: ${p\choose q} = {p-1\choose q}+{p-1\choose q-1}$.

Finally using $\langle M\rangle$ in $P(R_k|R_{\text{later}}B)$, we have the answer \begin{equation} P(R_k|R_{\text{later}}B) = \frac{M}{N-n-k}\times\frac{{N-1 \choose n-k} - {N-M \choose n-k}}{{N \choose n-k} -{N-M \choose n-k}} \end{equation}