# How to choose a importance density for Jeffreys prior?

I want to draw Bayesian inference via importance sampling and I do not come up with a good idea of an importance density for $$p(\sigma)\sim\frac{1}{\sigma}.$$ Is there a way to sample from this distribution directly? I am not sure whether $$\frac{1}{z}, z\sim\mathcal{U}[0,1]$$ will give me the desired property.

Update: I implemented the proposed importance function and computed the weights $$w_t=p(\sigma_t|\mathbf{y})\Big/\left\{\frac{1}{2a}\mathbb{I}_{[0,a]}(\sigma_t)+\frac{a}{2}\mathbb{I}_{[a,\infty)}(\sigma_t)\sigma_t^{-2}\right\}$$ for a sample of $m=1.000.000$ draws. Hereby I computed the log likelihoods $\omega_{log}$, substracted the maximum $\omega_{log}$ and transformed $\omega=\exp(\omega_{log}-\omega_\max)$. What can I learn from the fact, that roughly $0.98 %$ percent of the weights are $0$? I suppose that in this case the choice of the importance sample is not sensitive, as the sample size does not really matter but only a couple of observations drive the whole result?

• Well, I want to implement Importance sampling for a model that uses the Jeffreys prior. Therefore I am searching for a way to come up with a suitable importance density $q(\sigma)$ in order to compute posterior integrals $\int h(\sigma)p(\sigma|Y)d\sigma=\int h(\sigma)\frac{p(\sigma|Y)}{q(\sigma)}q(\sigma)d\sigma$. – muffin1974 Nov 23 '15 at 10:57

The short answer to your question is that it is impossible to simulate from $p(\sigma)=1/\sigma$ as this is not a probability density but a measure with infinite mass.

Since you also mention Jeffreys and importance in the same sentence, it may however be that you are actually asking about simulating a posterior associated with $p(\sigma)=1/\sigma$ as the Jeffreys prior. In this case, a fat tailed importance sampler would do. For instance, if the posterior accumulates mass around $a$, e.g., if $a$ is roughly the mode. you could use $$\frac{1}{2}\mathcal{U}[0,a]+\frac{1}{2}1\big/\mathcal{U}[0,1/a]$$ as an importance function. This means

1. Simulate a sample $\sigma_1,\ldots,\sigma_M$ from this distribution;
2. Weight the above simulations by $$w_t=p(\sigma_t|\mathbf{y})\Big/\left\{\frac{1}{2a}\mathbb{I}_{[0,a]}(\sigma_t)+\frac{a}{2}\mathbb{I}_{[a,\infty)}(\sigma_t)\sigma_t^{-2}\right\}$$

since the density of an inverse uniform is $$f(z)=a\mathbb{I}_{(0,1/a)}(1/z)\frac{\text{d}u}{\text{d}z}=a\mathbb{I}_{z>a}\frac{1}{z^2}$$

Remember, always check that the resulting importance weights have finite variance!

• I just realized that I do not know what to use as $q(\sigma)$ under this importance density...can you give a comment on that? – muffin1974 Nov 23 '15 at 12:37
• Is there a theoretical framework recommending this idea? I get the point with the fat tails, however, I would never come up with this distribution. Is it somewhat standard in importance sampling literature? – muffin1974 Nov 24 '15 at 10:00
• By the way, is the weighting function really correct? Why does the second term include a \$\sigma^{-2} and the first one not? – muffin1974 Nov 24 '15 at 10:08
• @muffin1974: No to all questions:1. this is a default proposal if you cannot come up with anything else; 2. it has no particular theoretical property that I know of; 3. it is not standard. I made this proposal to illustrate importance sampling not to suggest a universal solution. – Xi'an Nov 24 '15 at 10:10