Kullback-Leibler divergence is not symmetric but why mutual information is symmetric? As we know Kullback-Leibler divergence is not symmetric and mutual information is the KL divergence between P(X,Y) and P(X)*P(Y).So my question is that why mutual information is symmetric which is a kind of KL divergence ?
 A: The mutual information (MI) between $p(x)$ and $p(y)$ can be interpreted as the KL-divergence of the marginal product $p(x)p(y)$ to the joint distribution $p(x,y)$. In words, this is saying how different the product of marginals is from the joint. If the joint is very similar to the product of marginals, that means $x$ and $y$ must be (nearly) independent of one another, and so there is little MI. If they are very different from one another, however, then the joint distribution of $x$ & $y$ is not well approximated by the product of their marginals, and so $x$ & $y$ must share substantial MI. 
Mutual information is symmetric between $x$ and $y$, because the KL-divergence is computed between $p(x)p(y)$ and $p(x,y)$. Note that the asymmetry of KL-divergence means that (in general) $D_{KL}(P||Q)\neq D_{KL}(Q||P)$. In the formula for MI, this means that you would get a different answer if, instead of the divergence of $p(x)p(y)$ w.r.t. $p(x,y)$, you computed the divergence of $p(x,y)$ w.r.t. $p(x)p(y)$. In other words, you can switch $x$ and $y$ around with no effect on the result, which is the symmetry of MI you were referring to, but you cannot swap the product of marginals with the joint, which is the asymmetry of the KL-divergence.  
In other words, MI is computed between $x$ and $y$, by computing the KL-divergence of $p(x)p(y)$ to $p(x,y)$. The symmetry of MI is between $x$ and $y$, rather than between the arguments of the KL-divergence. The KL-divergence is still asymmetric (as always) w.r.t. its arguments, i.e. between $p(x)p(y)$ and $p(x,y)$.
A: The mutual information $I(X;Y)$ is the relative entropy(or KL divergence) between the joint distribution and the product distribution $p(x)p(y)$.
$$I(X;Y)=\sum_{x\in \mathcal{X}}\sum_{y\in \mathcal{Y}}p(x,y)log\frac{p(x,y)}{p(x)p(y)}=D(p(x,y)||p(x)p(y))$$.
We can see that both the joint distribution $p(x,y)$ and product distribuion $p(x)p(y)$ are symmetric, then the mutual information is symmetric for $x$ and $y$. But note that the symmetry doesn't work for $p(x,y)$ and $p(x)p(y)$: $D(p(x,y)||p(x)p(y))\neq D(p(x)p(y)||p(x,y))$
While for the normal relative entropy(or KL divergence) $D(p(x)||p(y))=\sum_{x\in \mathcal{X}}p(x)log\frac{p(x)}{p(y)}$, we can see that it is not symmetric for $x$ and $y$.
In addition, the mutual information is the reduction in the uncertainty of $X$ due to the knowledge of $Y$: $I(X;Y)=H(Y)-H(Y|X)$, and by symmetry, it also follows that  $I(Y;X)=H(X)-H(X|Y)$. Thus, $X$ says as much about $Y$ as $Y$ says about $X$.

Refernece:
Elements of information theory
