2
$\begingroup$

I want to draw Bayesian inference via importance sampling and I do not come up with a good idea of an importance density for $$p(\sigma)\sim\frac{1}{\sigma}.$$ Is there a way to sample from this distribution directly? I am not sure whether $$\frac{1}{z}, z\sim\mathcal{U}[0,1]$$ will give me the desired property.

Update: I implemented the proposed importance function and computed the weights $$w_t=p(\sigma_t|\mathbf{y})\Big/\left\{\frac{1}{2a}\mathbb{I}_{[0,a]}(\sigma_t)+\frac{a}{2}\mathbb{I}_{[a,\infty)}(\sigma_t)\sigma_t^{-2}\right\}$$ for a sample of $m=1.000.000$ draws. Hereby I computed the log likelihoods $\omega_{log}$, substracted the maximum $\omega_{log}$ and transformed $\omega=\exp(\omega_{log}-\omega_\max)$. What can I learn from the fact, that roughly $0.98 %$ percent of the weights are $0$? I suppose that in this case the choice of the importance sample is not sensitive, as the sample size does not really matter but only a couple of observations drive the whole result?

$\endgroup$
  • $\begingroup$ Well, I want to implement Importance sampling for a model that uses the Jeffreys prior. Therefore I am searching for a way to come up with a suitable importance density $q(\sigma)$ in order to compute posterior integrals $\int h(\sigma)p(\sigma|Y)d\sigma=\int h(\sigma)\frac{p(\sigma|Y)}{q(\sigma)}q(\sigma)d\sigma$. $\endgroup$ – muffin1974 Nov 23 '15 at 10:57
6
$\begingroup$

The short answer to your question is that it is impossible to simulate from $p(\sigma)=1/\sigma$ as this is not a probability density but a measure with infinite mass.

Since you also mention Jeffreys and importance in the same sentence, it may however be that you are actually asking about simulating a posterior associated with $p(\sigma)=1/\sigma$ as the Jeffreys prior. In this case, a fat tailed importance sampler would do. For instance, if the posterior accumulates mass around $a$, e.g., if $a$ is roughly the mode. you could use $$\frac{1}{2}\mathcal{U}[0,a]+\frac{1}{2}1\big/\mathcal{U}[0,1/a]$$ as an importance function. This means

  1. Simulate a sample $\sigma_1,\ldots,\sigma_M$ from this distribution;
  2. Weight the above simulations by $$w_t=p(\sigma_t|\mathbf{y})\Big/\left\{\frac{1}{2a}\mathbb{I}_{[0,a]}(\sigma_t)+\frac{a}{2}\mathbb{I}_{[a,\infty)}(\sigma_t)\sigma_t^{-2}\right\}$$

since the density of an inverse uniform is $$ f(z)=a\mathbb{I}_{(0,1/a)}(1/z)\frac{\text{d}u}{\text{d}z}=a\mathbb{I}_{z>a}\frac{1}{z^2} $$

Remember, always check that the resulting importance weights have finite variance!

$\endgroup$
  • $\begingroup$ I just realized that I do not know what to use as $q(\sigma)$ under this importance density...can you give a comment on that? $\endgroup$ – muffin1974 Nov 23 '15 at 12:37
  • $\begingroup$ Is there a theoretical framework recommending this idea? I get the point with the fat tails, however, I would never come up with this distribution. Is it somewhat standard in importance sampling literature? $\endgroup$ – muffin1974 Nov 24 '15 at 10:00
  • $\begingroup$ By the way, is the weighting function really correct? Why does the second term include a $\sigma^{-2} and the first one not? $\endgroup$ – muffin1974 Nov 24 '15 at 10:08
  • 1
    $\begingroup$ @muffin1974: No to all questions:1. this is a default proposal if you cannot come up with anything else; 2. it has no particular theoretical property that I know of; 3. it is not standard. I made this proposal to illustrate importance sampling not to suggest a universal solution. $\endgroup$ – Xi'an Nov 24 '15 at 10:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.