1
$\begingroup$

Given a known density $p(x)$, I'd like to generate samples according to $q(x) \propto p(x) f(x)$, where $f(x)$ is some probability function, $\forall x f(x) \in [0, 1]$, e.g., a sigmoid function. One way to generate such samples is by using a type of rejection sampling where we first generate $x \sim p$ then accept / reject based on the value of $f(x)$.

However, I'd like to use $q(x)$ as a proposal density in the context of importance sampling to generate samples that are rare under $p$. So, this rejection sampling approach would be very inefficient and essentially defeat the purpose of using importance sampling.

Is there a more efficient way to sample according to $q(x)$ as above, somehow using the fact that $f(x)$ is a probability function, e.g., a sigmoid function? Any insight would be greatly appreciated.

$\endgroup$
4
  • $\begingroup$ Is $f$ a probability density or a cumulative density? $\endgroup$
    – jbowman
    Commented Jun 21, 2022 at 1:27
  • $\begingroup$ @jbowman For the specific problem I'm looking at, it's more a scoring function whose output is in [0, 1] e.g., $f(x)$ being how likely $x$ to meet some criteria. Perhaps I shouldn't have called $f$ a probability function. $\endgroup$
    – kykim
    Commented Jun 21, 2022 at 5:28
  • 1
    $\begingroup$ There are many potential algorithms that might be used. If you can find a suitable envelope for $q$ there's accept-reject (and a variety of potential ways to obtain some). If you have a suitable approximation to use in a proposal distribution and don't mind dependence in your generated values you may be able to use Metropolis-Hastings. etc etc $\endgroup$
    – Glen_b
    Commented Jun 22, 2022 at 0:48
  • $\begingroup$ "Probability function" has no particular meaning in this setting. For rare events, cross-entropy is a way to build a better proposal than $p(\cdot)$ $\endgroup$
    – Xi'an
    Commented Jun 22, 2022 at 9:20

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.