3
$\begingroup$

What does it mean for a probability density function $f(x)$ to have the following property? $$I= 1+\int_{x=0}^{\infty}x^2 \left(\frac{f'(x)^2}{f(x)}-f''(x)\right)dx>0$$

This comes from minimization of Kullback Leibler divergence, when you want to make the coefficient of the second term of the Taylor expansion of $D_{KL}$ between $f(x)$ and $g(x)$ be positive, where $g(x)=(1-a)g(x(1-a))$ and $0<a<1$,

$$\frac{\partial}{\partial a^2}D_{KL}(g(x)||f(x))\Bigg|_{r=0}>0$$

I have tried a lot to simplify this condition and see what it means (in terms of moments of $f(x)$, etc), but no luck yet. The only simplifications I found are:

\begin{align} I&= 1-\int_{x=0}^{\infty}x^2 f(x) \frac{d^2 \log{f(x)}}{dx^2}dx\\ &= 1-\int_{x=0}^{\infty} \left(x^2 f'(x)\right)' { \log{f(x)}}dx \end{align}

Do you have any idea?

P.S. I found that it is one of the regulatory conditions for fisher information metric.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Could you elaborate on the minimization problem? It sounds like you are doing KL divergence minimization with some additional constraints? $\endgroup$
    – Alex R.
    Commented Jul 11, 2016 at 12:46
  • $\begingroup$ @AlexR. Sure, I added more details to clarify this. $\endgroup$
    – Sus20200
    Commented Jul 13, 2016 at 11:34
  • $\begingroup$ The motivation for this still seems strange. KL divergence is minimized when $f=g$. Heuristically, you could perturb this to $f(x)=g(x)+\epsilon(x)$, where $\epsilon(x)$ is some "small" function, to ensure you get a positive Hessian (which might be zero at the minimum). In general there are going to be infinitely many solutions to this problem. $\endgroup$
    – Alex R.
    Commented Jul 14, 2016 at 16:19

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.