On a mistake computing the Kullback Liebler Information Criterion THE FRAMEWORK:
Let $X_1$ be an observation from a normal random variable with mean zero and variance $\sigma^2$ and lets call the PDF $f(x)$.
I want to minimize the Kullback Liebler Information criterion between a PDF $g(x, \beta) $ of a zero mean normal random variable with variance $\beta \sigma^2$ and
 $f(x)$. The minimization is with respect to $\beta$.
The Kullback Liebler Information criterion is defined as t
$$I(f: g, \beta):= E [ \log(f(X_1)/g(X_1, \beta)   )]$$
THE MOTIVATION:
The motivation for doing this is that Akaike showed that the maximum likelihood estimator $\hat{\beta}$ of a model that assumes $g$ as the parametric distribution generating the observations is a natural estimator for the value $\beta^*$ that minimizes the Kullback Liebler information criterion.
THE PROBLEM:
I wanted to do this simple calculation because I expected $\beta^* = 1$ (in this way the two distributions would be equal so their "distance" is minimized). But performing the computations I obtain
$$ E \left[  \frac{1}{2} \log \beta X_1^2 - \frac{1}{2} \log \sigma^2 - \frac{\beta X_2^2 + \sigma^2}{2 \sigma^2 \beta}   \right] $$
and it seems to me that the $\beta^* $ that minimizes this tends to minus infinity. Where is my mistake?
 A: I am not so sure why there exist both "$X_1$" and "$X_2$" in your expression of KL information, but I think the following solution should be right.
According to the expression of KL information, we have
$$I(f:g)=\int_x g(x)\log\left[\frac{f(x)}{g(x)}\right] dx=\int_xg(x)\left[\log(f(x))-\log(g(x))\right] dx.$$
Here "$\log$" refers to natural logarithms. Take the expression $f(x)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{x^2}{2\sigma^2}}$ and $g(x)=\frac{1}{\sqrt{2\pi\beta\sigma^2}}e^{-\frac{x^2}{2\beta\sigma^2}}$ into the above expression, we have:
$$I(f:g)=\int_xg(x)\left[-\frac{1}{2}\log(2\pi\sigma^2)-\frac{x^2}{2\sigma^2}+\frac{1}{2}\log(2\pi\beta\sigma^2)+\frac{x^2}{2\beta\sigma^2}\right] dx.$$
Due to the Gaussian distribution, we know that $\int_xg(x)x^2=\beta\sigma^2$ and $\int_xg(x)=1$. So the KL information can be expressed as:
$$I(f:g)=\frac{1}{2}\log\left(\frac{2\pi\beta\sigma^2}{2\pi\sigma^2}\right)+\frac{\beta\sigma^2}{2\beta\sigma^2}-\frac{\beta\sigma^2}{2\sigma^2}=\frac{1}{2}\log\beta+\frac{1}{2}-\frac{\beta}{2}$$
Take derivation with respect to $\beta$ and set the result to 0, we can obtain that
$$\frac{1}{2\beta}-\frac{1}{2}=0\rightarrow\beta=1$$
I think this is exactly what you want.
