2
$\begingroup$

I never actually use Jeffreys priors really, generally using weakly informative priors instead. But one thing I've never quite understood about Jeffreys priors are where the Fisher information comes from.

For example, this page states that "construction is based on the Fisher information function of a model." which inclines me to think it's not based on the observed data, but on a set of plausible parameter values one might observe. But that contradicts the fact that it's an uninformative prior. On the other hand, I've never seen Jeffreys prior described as data dependent so using the Fisher information of the observed data doesn't quite make sense either.

$\endgroup$
  • 2
    $\begingroup$ Sir Harold Jeffreys: hence Jeffreys prior, Jeffreys' prior, Jeffreys's prior are all defensible, but never Jeffrey's. $\endgroup$ – Nick Cox Aug 31 '18 at 23:06
4
$\begingroup$

The standard definition of the Fisher information of a model with density $f(\cdot|\theta)$ is the function of $\theta$ $$\mathfrak{I}(\theta)=\mathbb{E}_\theta\left[\text{var} \frac{\partial \log f(X|\theta)}{\partial\theta}\right]=-\mathbb{E}_\theta\left[\frac{\partial^2 \log f(X|\theta)}{\partial\theta\partial\theta^\text{T}}\right]$$ so there is no data involved in this concept. It is a function of the parameter that can be turned into a Jeffreys prior measure $|\mathfrak{I}(\theta)|^{1/2}$. The observed information is$$\frac{\partial^2 \log f(x^\text{obs}|\theta)}{\partial\theta\partial\theta^\text{T}}$$and is never used for prior determination, being dependent on the data and all that. In cases when $\mathfrak{I}(\theta)$ cannot be computed analytically, a Monte Carlo version is the average $$\hat{\mathfrak{I}}(\theta)=\frac{1}{N}\sum_{i=1}^N \frac{\partial^2 \log f(x_i^\text{sim}|\theta)}{\partial\theta\partial\theta^\text{T}}$$ which still does not involve the observed information.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.