# Conditional entropy of sum of random variables

How can be proven that for random variables $A$ and $B$, and $C = A + B$,

$$H(C\mid A) = H(B\mid A).$$

Also, would it be possible to determine if $H(C)$ would be greater than $H(A)$?

• any ideas will be appreciated. Commented Oct 26, 2012 at 1:22
• It is a homework, isn't it? Just think about what happens when $A=a$. What is the difference between $C|A=a$ and $B|A=a$. Anyway, looking at en.wikipedia.org/wiki/Convolution may help (if two probabilities have densities, the density of their sum is just a convolution of their densities). Commented Nov 3, 2012 at 0:15
• Differential or Shannon entropy? The sum of absolutely continuous variables need not even be absolutely continuous. Given the poor behavior of differential entropy under sums, I would doubt there is such a relationship for differential entropy. Commented Feb 3, 2018 at 19:21
• Some observations. The problem seems equivalent to showing $H(A,A+B)=H(A,B)$. Also, is there anything special about addition here? Would the relationship $H(A,AB)=H(A,B)$ also hold under suitable conditions? Commented Feb 3, 2018 at 19:24

This is an elementary exercise, when the random variables take only a numerable number of values: $$A(\omega)\in \mathcal{A}$$, $$B(\omega)\in \mathcal{B}$$ and $$C(\omega)\in \mathcal{C}$$. In that case:

$$H(C|A) = -\sum_{a\in\mathcal{A}}\sum_{c\in\mathcal{C}} p_A(a)p_{C|A}(c|a)\ln p_{C|A}(c|a), \tag{*}\label 1$$

Since $$C = A + B$$, we have that:

$$p_{C|A}(c|a) = p_{B|A}(c-a|a)$$

And substituting this into $$\eqref 1$$, the conditional entropy is:

$$H(C|A) = -\sum_{a\in\mathcal{A}}\sum_{c\in\mathcal{C}} p_A(a)p_{B|A}(c-a|a)\ln p_{B|A}(c-a|a), \tag{**}\label 2$$

restructuring the second sum over values of $$b = c-a$$, we have:

$$H(C|A) = -\sum_{a\in\mathcal{A}} p_A(a) \sum_{b\in\mathcal{B}} p_{B|A}(b|a)\ln p_{B|A}(b|a) = -\sum_{a\in\mathcal{A}} p_A(a) H(B|a) = H(B|A).$$