1
$\begingroup$

Consider $m$ channels indexed by $i$ with $1 \leq i \leq m$. The input alphabets are from the same finite set $\mathcal{X}$. Let $\pi$ denote a probability distribution on $\mathcal{X}$. Define the function \begin{equation} f_i(\pi)=I(X:Y_i) \end{equation} where $X \sim \pi$ and given $X=x$, $Y_i$ is generated as $Y_i|X=x \sim Y|i,x$, that is we send the input $X$ to the $i^{th}$ channel and observe the output $Y_i$. $I(.,.)$ is the mutual information between two random variables. Further, define $f(\pi)=max_i f_i(\pi)$.

Now suppose we saw an output $Y_j$ from a fixed channel $j$. Assuming a prior $\pi_0$ on $\mathcal{X}$ we denote the posterior after seeing $Y_j$ as $\pi_{1,j}$. Is it true that: \begin{equation} f(\pi_0) \geq E[f(\pi_{1,j})] \end{equation} for all $j \in [m]$? Note the expectation is taken over the randomness of $Y_j$ and $X$.

It is easily seen that : \begin{equation} f_i(\pi_0) \geq E[f_i(\pi_{1,j})] \end{equation} by concavity of mutual information and the fact the expectation of posterior is just the prior. Unfortunately the $max$ operation destroys this concavity for $f$. But the inequality seems intuitive in that the maximum information you can extract about the input from any channel decreases as the number of observations accumulate.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.