4
$\begingroup$

KL divergence between two Gaussian distributions denoted by $\mathcal{N}(\mathbf \mu_1, \mathbf \Sigma_1)$ and $\mathcal{N}(\mathbf \mu_2, \mathbf \Sigma_2)$ is available in a closed form as:

$$\mathbb{KL}= \frac{1}{2}\left[\log\frac{|\Sigma_2|}{|\Sigma_1|} - d + \text{tr} \{ \Sigma_2^{-1}\Sigma_1 \} + (\mu_2 - \mu_1)^T \Sigma_2^{-1}(\mu_2 - \mu_1)\right]$$

from: KL divergence between two multivariate Gaussians

My question is assuming $\mathbf \mu_2=\mathbf \mu_1 +\mathbf \epsilon_\mu$ and $\mathbf \Sigma_2=\mathbf \Sigma_1 +\epsilon_\Sigma$ (each element of mean and covariance are perturbed by a small amount), and $\mathbf \Sigma_1$ and $\mathbf \Sigma_2$ have Toeplitz form, can the above formulla be simplified? If closed-from solution can not be obtained, can someone upper bound KL for this setup?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ The means aren't that interesting; the quadratic form just sees the difference in means become epsilon. You'd have to likely be more specific about how the covariance is perturbed. I'm not confident that just adding gaussian noise to a covariance matrix preserves some of its properties, namely positive definiteness. $\endgroup$
    – Demetri Pananos
    Mar 3, 2020 at 2:36
  • 1
    $\begingroup$ @Demetri The description given here does not involve "adding Gaussian noise," as we can surmise from the statement that both $\Sigma_i$ are Toeplitz. In this context it would be safe to assume the perturbations are deterministic and arranged so that the question makes sense; that is, the $\Sigma_i$ are positive semi-definite symmetric matrices. That's a pretty specific perturbation. $\endgroup$
    – whuber ♦
    Mar 3, 2020 at 13:32

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.