For two perfectly correlated Gaussian variables, the mutual information between them, and thus the KL divergence between the product of the marginal distributions and the joint distribution, is infinity. I have read that KL divergence is always finite if Q is continuous in respect to P, but I am having a hard time understanding why, and how to know what the maximum KL divergence between the product of the marginals and the joint is for any given marginal distributions. I have a feeling that the maximum possible mutual information between two distributions is related to the KL divergence of their marginal distributions - i.e. a uniformly distributed variable cannot have infinite mutual information with a Gaussian distributed variable, and that is because the KL divergence between two two marginals is not 0. However, if I know the KL divergence of two marginal distributions, is it possible to calculate the maximum possible mutual information between them?
I hope this isn't a stupid question, but I know there must be a relationship and I can't find anything about it readily.