Given the following hierarchical model, $$ X \sim {\mathcal N}(\mu,1), $$ and, $$ \mu \sim {\rm Laplace}(0, c) $$ where $\mathcal{N}(\cdot,\cdot)$ is a normal distribution. Is there a way to get an exact expression for the Fisher information of the marginal distribution of $X$ given $c$. That is, what is the Fisher information of: $$ p(x | c) = \int p(x|\mu) p(\mu|c) d\mu $$ I can get an expression for the marginal distribution of $X$ given $c$, but differentiating w.r.t. $c$ and then taking expectations seems very difficult. Am I missing something obvious? Any help would be appreciated.
$\begingroup$
$\endgroup$
6
-
$\begingroup$ I had a try at it myself, but it is beyond my abilities. Absolute value functions ruin everything! You are basically stuck with numerical methods. $\endgroup$– probabilityislogicMar 28, 2011 at 12:40
-
3$\begingroup$ @probability You can obtain an expression for the integrand simply by splitting the integral into regions $\mu \ge 0$ and $\mu \lt 0$; no absolute values are needed. But the result is a messy rational function of $x$, $exp(-x^2)$, and error functions, and so is unlikely to be integrable in closed form. $\endgroup$– whuber ♦Mar 28, 2011 at 17:23
-
1$\begingroup$ @whuber - that is what I meant by "hopeless". Not that the integral is impossible, but the fisher information is impossible. Because you have to take the expected value over $X$ of a ratio of two of these types of integral $\endgroup$– probabilityislogicMar 28, 2011 at 22:02
-
1$\begingroup$ A lower bound for the Fisher information in this case is $1/(1+2c^2)$. Is it possible to get a tighter uppper bound on the Fisher information than the general $1 + 1/c^2$? $\endgroup$– emakalicApr 3, 2011 at 23:56
-
$\begingroup$ While an analytic solution would be a challenge in terms of human tractability (outside of a mathematician discipline), is there receptivity to an approximate computational solution? One could make a stochastic simulation and then look at approximations for the fit. $\endgroup$– EngrStudentFeb 13, 2016 at 13:18
|
Show 1 more comment
2 Answers
$\begingroup$
$\endgroup$
There is no closed-form analytic expression for the Fisher information for the hierarchical model you give. In practice, Fisher information can only be computed analytically for exponential family distributions. For exponential families, the log-likelihood is linear in the sufficient statistics, and the sufficient statistics have known expectations. For other distributions, the log-likelihood does not simplify in this way. Neither the Laplace distribution nor the hierarchical model are exponential family distributions, so an analytic solution will be impossible.
answered Dec 4, 2016 at 3:54
$\begingroup$
$\endgroup$
1
The two of the Normal and Laplace are from the exponential family. If you can write the distribution in the exponential form then the fisher information matrix is the second gradient of the log-normalizer of the exponential family.
-
1$\begingroup$ I don't think the usual two-parameter Laplace is in the exponential family. If the location parameter is known then it will be in the exponential family, but I believe that $\frac12 \exp(-|x-\mu|)$ cannot be written in exponential family form. $\endgroup$– Glen_bDec 6, 2016 at 8:41