# KL divergence between two multivariate Gaussians with close means and variances

KL divergence between two Gaussian distributions denoted by $$\mathcal{N}(\mathbf \mu_1, \mathbf \Sigma_1)$$ and $$\mathcal{N}(\mathbf \mu_2, \mathbf \Sigma_2)$$ is available in a closed form as:

$$\mathbb{KL}= \frac{1}{2}\left[\log\frac{|\Sigma_2|}{|\Sigma_1|} - d + \text{tr} \{ \Sigma_2^{-1}\Sigma_1 \} + (\mu_2 - \mu_1)^T \Sigma_2^{-1}(\mu_2 - \mu_1)\right]$$

My question is assuming $$\mathbf \mu_2=\mathbf \mu_1 +\mathbf \epsilon_\mu$$ and $$\mathbf \Sigma_2=\mathbf \Sigma_1 +\epsilon_\Sigma$$ (each element of mean and covariance are perturbed by a small amount), and $$\mathbf \Sigma_1$$ and $$\mathbf \Sigma_2$$ have Toeplitz form, can the above formulla be simplified? If closed-from solution can not be obtained, can someone upper bound KL for this setup?

• The means aren't that interesting; the quadratic form just sees the difference in means become epsilon. You'd have to likely be more specific about how the covariance is perturbed. I'm not confident that just adding gaussian noise to a covariance matrix preserves some of its properties, namely positive definiteness. Mar 3, 2020 at 2:36
• @Demetri The description given here does not involve "adding Gaussian noise," as we can surmise from the statement that both $\Sigma_i$ are Toeplitz. In this context it would be safe to assume the perturbations are deterministic and arranged so that the question makes sense; that is, the $\Sigma_i$ are positive semi-definite symmetric matrices. That's a pretty specific perturbation.
– whuber
Mar 3, 2020 at 13:32