20
$\begingroup$

Given the following hierarchical model, $$ X \sim {\mathcal N}(\mu,1), $$ and, $$ \mu \sim {\rm Laplace}(0, c) $$ where $\mathcal{N}(\cdot,\cdot)$ is a normal distribution. Is there a way to get an exact expression for the Fisher information of the marginal distribution of $X$ given $c$. That is, what is the Fisher information of: $$ p(x | c) = \int p(x|\mu) p(\mu|c) d\mu $$ I can get an expression for the marginal distribution of $X$ given $c$, but differentiating w.r.t. $c$ and then taking expectations seems very difficult. Am I missing something obvious? Any help would be appreciated.

$\endgroup$
  • $\begingroup$ I had a try at it myself, but it is beyond my abilities. Absolute value functions ruin everything! You are basically stuck with numerical methods. $\endgroup$ – probabilityislogic Mar 28 '11 at 12:40
  • 3
    $\begingroup$ @probability You can obtain an expression for the integrand simply by splitting the integral into regions $\mu \ge 0$ and $\mu \lt 0$; no absolute values are needed. But the result is a messy rational function of $x$, $exp(-x^2)$, and error functions, and so is unlikely to be integrable in closed form. $\endgroup$ – whuber Mar 28 '11 at 17:23
  • 1
    $\begingroup$ @whuber - that is what I meant by "hopeless". Not that the integral is impossible, but the fisher information is impossible. Because you have to take the expected value over $X$ of a ratio of two of these types of integral $\endgroup$ – probabilityislogic Mar 28 '11 at 22:02
  • 1
    $\begingroup$ A lower bound for the Fisher information in this case is $1/(1+2c^2)$. Is it possible to get a tighter uppper bound on the Fisher information than the general $1 + 1/c^2$? $\endgroup$ – emakalic Apr 3 '11 at 23:56
  • $\begingroup$ While an analytic solution would be a challenge in terms of human tractability (outside of a mathematician discipline), is there receptivity to an approximate computational solution? One could make a stochastic simulation and then look at approximations for the fit. $\endgroup$ – EngrStudent - Reinstate Monica Feb 13 '16 at 13:18
2
$\begingroup$

There is no closed-form analytic expression for the Fisher information for the hierarchical model you give. In practice, Fisher information can only be computed analytically for exponential family distributions. For exponential families, the log-likelihood is linear in the sufficient statistics, and the sufficient statistics have known expectations. For other distributions, the log-likelihood does not simplify in this way. Neither the Laplace distribution nor the hierarchical model are exponential family distributions, so an analytic solution will be impossible.

$\endgroup$
0
$\begingroup$

The two of the Normal and Laplace are from the exponential family. If you can write the distribution in the exponential form then the fisher information matrix is the second gradient of the log-normalizer of the exponential family.

$\endgroup$
  • $\begingroup$ I don't think the usual two-parameter Laplace is in the exponential family. If the location parameter is known then it will be in the exponential family, but I believe that $\frac12 \exp(-|x-\mu|)$ cannot be written in exponential family form. $\endgroup$ – Glen_b -Reinstate Monica Dec 6 '16 at 8:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.