# What is the distribution G (with given mean and variance) that has the minimum KL distance from normal distribution F?

I was hoping someone knows whether the following statement is true or not.

Suppose $F$ is a given normal distribution and $G$ is a distribution that has a given mean $\mu$ and variance $\sigma^2$ (but can otherwise be anything). Then, the distribution G that has the minimum KL distance from F is a normal distribution (of course, $G$ would be a normal distribution mean $\mu$ and variance $\sigma^2$).

Is this true and if so, is there a reference? Really appreciate any assistance.

## 1 Answer

Let me share my attempt in the real-valued case for simplicity. Since $$F$$ has a density $$f$$ and we want to minimize the KL to $$F$$ it is enough to look for distributions $$G$$ with some density $$g$$. We want to find $$g$$ such that \begin{align} g \in \arg\min &\int g(x) \log\left(\frac{g(x)}{f(x)}\right) dx \qquad \text{subject to }\\ \int g(x) dx = 1, \qquad &\int x g(x) dx = \mu, \qquad\int x^2 g(x) dx = \sigma^2 \end{align} for some fixed $$\mu, \sigma^2$$. This is a standard variational problem with integral constraints, and we know that for $$g$$ to be an extremum it must solve the Euler-Lagrange equation $$\begin{equation} \frac{\partial}{\partial g} \left[ g(x)\log\left(\frac{g(x)}{f(x)}\right) + \lambda g(x) + \phi x g(x) + \theta x^2 g(x)\right] = 0 \end{equation}$$ for some constants $$\lambda, \theta, \phi$$. Above display becomes $$\begin{equation} 1 + \log\left(\frac{g(x)}{f(x)}\right) + \lambda + \phi x + \theta x^2 = 0. \end{equation}$$ From here it follows that $$g(x) \propto f(x) \exp\left(-1-\lambda-\phi x - \theta x^2\right)$$. In particular, g will be a Gaussian density.