How can be proven that for random variables $A$ and $B$, and $C = A + B$,

$$H(C\mid A) = H(B\mid A).$$

Also, would it be possible to determine if $H(C)$ would be greater than $H(A)$?

  • $\begingroup$ any ideas will be appreciated. $\endgroup$ – whynot Oct 26 '12 at 1:22
  • $\begingroup$ It is a homework, isn't it? Just think about what happens when $A=a$. What is the difference between $C|A=a$ and $B|A=a$. Anyway, looking at en.wikipedia.org/wiki/Convolution may help (if two probabilities have densities, the density of their sum is just a convolution of their densities). $\endgroup$ – Piotr Migdal Nov 3 '12 at 0:15
  • $\begingroup$ Differential or Shannon entropy? The sum of absolutely continuous variables need not even be absolutely continuous. Given the poor behavior of differential entropy under sums, I would doubt there is such a relationship for differential entropy. $\endgroup$ – cantorhead Feb 3 '18 at 19:21
  • $\begingroup$ Some observations. The problem seems equivalent to showing $H(A,A+B)=H(A,B)$. Also, is there anything special about addition here? Would the relationship $H(A,AB)=H(A,B)$ also hold under suitable conditions? $\endgroup$ – cantorhead Feb 3 '18 at 19:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.