Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite

Suppose we have a $n$ pairs of mutually independent variables over $k$ outcomes and take a mixture distribution, what can we say about mutual information in the mixture? In particular I'm wondering about the upper bound on the mutual information. When $n=1$, the upper bound is $0$, and when $n=k^2$, the upper bound is vacuous, what can we say for $n<k^2$?

share|improve this question
1  
So, just to clarify, each $(X_i,Y_i) \sim F_i \times G_i$ pair has a different distribution and you want to consider a convex combination $H = \sum_i w_i (F_i \times G_i)$? – cardinal Jul 20 '12 at 1:25
1  
If you get a moment to confirm or correct my interpretation, that'd be great. Hopefully I'll find a bit of time to think about this a little bit. (+1 on the question.) – cardinal Jul 26 '12 at 19:11
1  
Yes, that's correct. – Yaroslav Bulatov Jul 26 '12 at 23:09

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.