I was wondering if this problem has been tackled in some way in the probability/functional analysis literature:

Given a pdf $f$ such that the expectation is zero and $\mu\in\mathbb R$, find the closest pdf $f^*_\mu$, in the sense of the Kullback-Leibler divergence from $f_\mu$ to $f$, such that the expectation of $f^*_\mu$ is $\mu$.

Or, equivalently, in an analysis-alike terminology:

Let $f:\mathbb{R}\rightarrow [0,\infty)$ be such that $\int f(x)\,dx=1$ and $\int x\,f(x)\,dx=0$ and let be the class of functions $\mathcal{C}_\mu=\{f_\mu:\mathbb{R}\rightarrow [0,\infty): \int f_\mu(x)\,dx=1, \,\int x\,f_\mu(x)\,dx=\mu\}$ for a given $\mu\in\mathbb{R}$. Find $$ f_\mu^*=\arg\min_{f_\mu\in\mathcal{C}_\mu} \int \log\left(\frac{f_\mu(x)}{f(x)}\right)f_\mu(x)\,dx. $$

I know about the trivial transformation $f_\mu(x)=f(x-\mu)$, but I was wondering as well if it was optimal under some circumstances (exponential families,...).

  • 2
    $\begingroup$ If $C_{\mu}$ is a set of parameterized pdfs by some $\theta$, then the problem might be solved in terms of standard nonlinear optimization (Karush-Kuhn-Tucker): $$ \theta^* = \arg\min_{\theta\in\Theta_\mu}\int \log\left(\frac{f_{\mu}(x;\theta)}{f(x)}\right)f_{\mu}(x;\theta) dx$$. For special cases, this could result in analytical results. If the set of parametric pdfs is dense for orignal $C_\mu$, then it is possible to find the original solution, I guess. $\endgroup$ – Karel Macek Apr 9 '15 at 8:54
  • $\begingroup$ Thanks for the comment @Karel, I guess if there was an analytical answer for a mixture of Gaussians it will be pleasantly enough :) $\endgroup$ – epsilone Apr 9 '15 at 21:15
  • 1
    $\begingroup$ This is exactly dealt in this paper: projecteuclid.org/euclid.aop/1176996454 $\endgroup$ – Ashok Aug 24 '15 at 11:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.