What is the distribution G (with given mean and variance) that has the minimum KL distance from normal distribution F? I was hoping someone knows whether the following statement is true or not.
Suppose $F$ is a given normal distribution and $G$ is a distribution that has a given mean $\mu$ and variance $\sigma^2$ (but can otherwise be anything). Then, the distribution G that has the minimum KL distance from F is a normal distribution (of course, $G$ would be a normal distribution mean $\mu$ and variance $\sigma^2$).
Is this true and if so, is there a reference? Really appreciate any assistance.
 A: Let me share my attempt in the real-valued case for simplicity. Since $F$ has a density $f$ and we want to minimize the KL to $F$ it is enough to look for distributions $G$ with some density $g$. We want to find $g$ such that
\begin{align}
g \in \arg\min &\int g(x) \log\left(\frac{g(x)}{f(x)}\right) dx \qquad \text{subject to }\\
\int g(x) dx = 1, \qquad &\int x g(x) dx = \mu, \qquad\int x^2 g(x) dx = \sigma^2
\end{align}
for some fixed $\mu, \sigma^2$. This is a standard variational problem with integral constraints, and we know that for $g$ to be an extremum it must solve the Euler-Lagrange equation
\begin{equation}
\frac{\partial}{\partial g} \left[ g(x)\log\left(\frac{g(x)}{f(x)}\right) + \lambda g(x) + \phi x g(x) + \theta x^2 g(x)\right] = 0
\end{equation}
for some constants $\lambda, \theta, \phi$. Above display becomes
\begin{equation}
1 + \log\left(\frac{g(x)}{f(x)}\right) + \lambda + \phi x + \theta x^2 = 0. 
\end{equation}
From here it follows that $g(x) \propto f(x) \exp\left(-1-\lambda-\phi x - \theta x^2\right)$. In particular, g will be a Gaussian density.
