I have to integrate a 2 dimensional function between a range which has two peaks. I am trying to combine two Gaussian functions to get a distribution which is close to the function so as to use importance sampling to make a better estimate with lower variance. Do i have to draw from both the samples equally? My second question is how do i combine the samples drawn from both these distributions?


1 Answer 1


If, when targeting the density $g(x)$ you use an importance density function of the form $$\alpha f_1(x)+(1-\alpha) f_2(x)$$ which is called a mixture distribution you can either

  1. simulate directly from the mixture, which amounts to simulate $$N_1\sim\mathcal{B}(n,\alpha)\quad x_1,\ldots,x_{N_1}\sim f_1\quad y_1,\ldots,y_{n-N_1}\sim f_2$$and use the importance weight$$\dfrac{g(x)}{\alpha f_1(x)+(1-\alpha) f_2(x)}$$for both the $x_i$'s and the $y_i$'s
  2. set $1\le n_1\le n-1$ in an arbitrary manner and simulate $$x_1,\ldots,x_{n_1}\sim f_1\quad y_1,\ldots,y_{n-n_1}\sim f_2$$and use the importance weight$$\dfrac{n g(x)}{n_1 f_1(x)+(n-n_1) f_2(x)}$$for both the $x_i$'s and the $y_i$'s

since both approaches are correct. The second approach is described in Owen and Zhou (2000, JASA) and extended in our adaptive multiple mixture paper.

  • $\begingroup$ What if the we only know $f_1$ and $f_2$ up to a constant where these two constants are different? $\endgroup$
    – TrungDung
    Sep 12, 2023 at 14:00
  • $\begingroup$ If unbiasedness is not relevant, one can use bridge sampling to estimate the constants at no additional computational cost. $\endgroup$
    – Xi'an
    Sep 12, 2023 at 16:01
  • $\begingroup$ @Xian: Given the Bayes theorem, of the denominator, p(y), is intractable, so for a complicated model, estimate that constant p(y) is not straightforward and could be time consuming. Or maybe I do not understand you? $\endgroup$
    – TrungDung
    Sep 13, 2023 at 9:58
  • $\begingroup$ See e.g. arxiv.org/abs/0801.3887, projecteuclid.org/journalArticle/…, conservancy.umn.edu/bitstream/handle/11299/199589/Technical Report 568 Reweighting Monte Carlo Mixtures.pdf $\endgroup$
    – Xi'an
    Sep 14, 2023 at 12:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.