1
$\begingroup$

So I'm trying to understand importance sampling. So far I have the algorithm implemented like this:

# Student's T target distribution for sake of example
target = stats.t(5, loc=5, scale=10)

particle_count = 100000
# Normal proposal distribution
proposal_dist = stats.norm(loc=0, scale=25)
# Take samples from proposal distribution
samples = proposal_dist.rvs(size=particle_count)
# Set weights according to ratio between target and proposal distributions
weights = target.pdf(samples) / proposal_dist.pdf(samples)
# Normalize
norm_weights = weights / np.sum(weights)

This gives the weights and calculating the expectation this way:

np.sum(norm_weights * samples)

gives around 5 as expected.

My question is how to generate "posterior" samples from the target distribution given the proposal distribution and the calculated weights.

$\endgroup$
1
  • $\begingroup$ make the target the unnormalized posterior (the likelihood times the prior). $\endgroup$
    – Taylor
    Aug 11, 2019 at 2:20

1 Answer 1

1
$\begingroup$

Importance sampling helps you estimate quantities for a target distribution (that you cannot generate samples from) using a proposal distribution (that you can generate samples from). Please see this great post by Xi'an on importance sampling.

To actually generate samples from the target distribution (or the posterior in a Bayesian analysis, where the posterior would now be your target distribution), we would need to use a more sophisticated algorithm like Metropolis-Hastings.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.