Shannon entropy for two random variables is symmetric: H(X,Y) = H(Y,X). How to prove it? Nielsen and Chuang's big book Qu. Comp. & Qu. Inf. says on pp. 506-507) it follows > from "the relevant definitions" that H(X,Y) = H(Y,X). I could not verify that.
 A: The definition of joint entropy is:
$$
H(X,Y) = -\sum_{\forall x \in X}\sum_{\forall y\in Y} P(X=x,Y=y)\log_2\big[P(X=x,Y=y)\big]
$$
You want to show that
$$ 
\begin{align*}
H(X,Y) &= H(Y,X) \\
-\sum_x \sum_y P(x,y) \log_2\big[P(x,y)\big] &= -\sum_y \sum_x P(y,x) \log_2\big[P(y,x)\big]
\end{align*}
$$
Two things differ between the left hand side and the right-hand side: the order of the summation and the order of the joint probability.
Does the summation order matter? No. Both $\sum_x \sum_y \ldots$ and $\sum_y \sum_x \ldots$ iterate over all possible pairs of $x$s and $y$s. Addition is commutative and associative, so the order is irrelevant.
Does the order within the joint probability matter? No. From the definition of joint probability, we know that $P(X=x \text{ and } Y=y)$ is the same as $P(Y=y \text{ and }X=x)$ because the and-ing is also commutative. If you'd prefer, you can take this one step further and derive this from the definition of conditional probability if you find that more "fundamental":
And we're done.
A: $$H(X,Y) = \mathrm{E}[\mathrm{I}(X,Y)] = \mathrm{E}[-\ln(\mathrm{P}(X,Y))]$$
Note 
$$P(X,Y)=P(Y,X)$$
Therefore we can write the equation backward.
