# Mutual information in a mixture of independent variables

Suppose we have a $n$ pairs of mutually independent variables over $k$ outcomes and take a mixture distribution, what can we say about mutual information in the mixture? In particular I'm wondering about the upper bound on the mutual information. When $n=1$, the upper bound is $0$, and when $n=k^2$, the upper bound is vacuous, what can we say for $n<k^2$?

-
So, just to clarify, each $(X_i,Y_i) \sim F_i \times G_i$ pair has a different distribution and you want to consider a convex combination $H = \sum_i w_i (F_i \times G_i)$? – cardinal Jul 20 '12 at 1:25
If you get a moment to confirm or correct my interpretation, that'd be great. Hopefully I'll find a bit of time to think about this a little bit. (+1 on the question.) – cardinal Jul 26 '12 at 19:11
Yes, that's correct. – Yaroslav Bulatov Jul 26 '12 at 23:09