Some context to my question: This is a part from a question about memoryless channel. The channel is given by $P_{Y\mid X}$, and $Q_Y$ is an arbitrary distribution on the channel outputs. in part of the solution for this question (the question itself is not relevant), they do the following move: $$max_{P_X}\sum_{x\in X}[P_X(x)D_{KL}(P_{Y\mid X=x}\mid\mid Q_Y)]= max_{x\in X}D_{KL}((P_{Y\mid X=x}\mid\mid Q_Y)$$

and they explain that it's true since

$max_{P_X}$ yields $P_X= 𝟙_{x=x_m}$ where $x\in X$ is maximizing the KL divergence

I don't really understand why it true, and I didn't understand the explanation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.