# Mutual Information decay

Consider $$m$$ channels indexed by $$i$$ with $$1 \leq i \leq m$$. The input alphabets are from the same finite set $$\mathcal{X}$$. Let $$\pi$$ denote a probability distribution on $$\mathcal{X}$$. Define the function $$$$f_i(\pi)=I(X:Y_i)$$$$ where $$X \sim \pi$$ and given $$X=x$$, $$Y_i$$ is generated as $$Y_i|X=x \sim Y|i,x$$, that is we send the input $$X$$ to the $$i^{th}$$ channel and observe the output $$Y_i$$. $$I(.,.)$$ is the mutual information between two random variables. Further, define $$f(\pi)=max_i f_i(\pi)$$.

Now suppose we saw an output $$Y_j$$ from a fixed channel $$j$$. Assuming a prior $$\pi_0$$ on $$\mathcal{X}$$ we denote the posterior after seeing $$Y_j$$ as $$\pi_{1,j}$$. Is it true that: $$$$f(\pi_0) \geq E[f(\pi_{1,j})]$$$$ for all $$j \in [m]$$? Note the expectation is taken over the randomness of $$Y_j$$ and $$X$$.

It is easily seen that : $$$$f_i(\pi_0) \geq E[f_i(\pi_{1,j})]$$$$ by concavity of mutual information and the fact the expectation of posterior is just the prior. Unfortunately the $$max$$ operation destroys this concavity for $$f$$. But the inequality seems intuitive in that the maximum information you can extract about the input from any channel decreases as the number of observations accumulate.