Relationship between KL divergence and correlation

Question

I know KL divergence tries to measure how different 2 probability distributions are.

I know high correlation values between 2 sets of variables imply they are highly dependent on each other.

Will the probability distributions associated with both sets of variables have low KL divergence between them, i.e.: will they be similar?

Answer 1

Not necessarily, e.g. $Y=cX$ and $X\sim N(0,1),\ c>0$, which means $Y\sim N(0,c^2)$. The KL divergence between two univariate normals can be calculated as laid out in here, and yields: $$KL(p_x||p_y)=2\log c+\frac{1}{2c^2}-{1\over2}$$

This can be arbitrarily large as $c$ changes but the correlation is always $1$.