# Can mutual information gain value be greater than 1

I have a very basic doubt. Sorry if this irritates few. I know that Mutual Information value should be greater than 0, but should it be less than 1 ? Is it bounded by any upper value ?

Thanks, Amit.

Yes, it does have an upper bound, but not 1.

The mutual information (in bits) is 1 when two parties (statistically) share one bit of information. However, they can share a arbitrary large data. In particular, if they share 2 bits, then it is 2.

The mutual information is bounded from above by the Shannon entropy of probability distributions for single parties, i.e. $I(X,Y) \leq \min \left[ H(X), H(Y) \right]$ .

• If the two parties $X,Y$ are binary variables i.e. each has only two possible outcomes {0,1}, then entropies $H(X), H(Y)$ max out at $1$ when $P(X)=0.5$ and $P(Y)=0.5$. Thus, maximum mutual information for two binary variables is $1$ – Akseli Palén Jul 4 at 14:43