This is my sampling algorithm

Let p(x) be a discrete distribution and f(x) be some function on real numbers. Consider

  1. $\textbf{Generate Samples from } p(x): \text{Initially, you sample } x_1, x_2, \ldots, x_n \text{ from the probability distribution } p(x).$ $ \text{Sample } x_i \text{ from } p(x), \text{ for } i = 1, 2, \ldots, n. $

  2. $\textbf{Reweigh Samples Using } f(x): \text{After generating these samples, reweigh them using the weights } \frac{f(x_i)}{\sum_{j=1}^n f(x_j)}$ $ \text{Weight for each sample } x_i: \frac{f(x_i)}{\sum_{j=1}^n f(x_j)}. $ Resample a point from this sampled set based on these weights.

What would be the marginal distribution of a point obtained using these two steps? I feel that it should be $\frac{p(x)f(x)}{\sum_{x'}p(x')f(x')}$ but I can't prove it?


2 Answers 2


The distribution of a point given all the first stage samples is $$ g(X = x_i \mid x_1, \ldots, x_n) = \frac{f(x_i)}{\sum_{j=1}^n f(x_j)}. $$ So the marginal distribution for finite $n$ is $$ \int \cdots \int g(X = x_i \mid x_1, \ldots, x_n)p(x_1) \cdots p(x_n) dx_1 \cdots dx_n. $$


It's not quite that, because your actual weight distribution (as written) has only the observed values in the denominator, not all values.

To take the extreme case, suppose $n=1$. You sample one value, $x_1$, from $p(x)$ and you reweight it using the weight $f(x_1)/f(x_1)=1$, so the marginal distribution is just $p(x)$

As $n$ increases, the distribution gets closer to being proportional to $p(x)f(x)$ because $$\frac{\sum_{i=1}^n f(x_j)}{\sum_{x'}f(x')}\stackrel{p}{\to} 1$$ by the law of large numbers


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.