0
$\begingroup$

How could I estimate an integral using Monte Carlo method when I have a mixture distribution? For example I want to estimate the below integral:

$I=\int_{0}^{1}f(x)dx$

And my distribution is:

$p(x)=\sum_{k=1}^{n}\lambda_kp_k(x)$

And $\langle I^N\rangle= \frac{1}{N}\sum_{i=1}^{N}\frac{f(x_i)}{p(x_i)}$ is a Monte Carlo estimate of $I$.

Is $p(x_i)$ the mixture density or conditional density?

$\endgroup$

2 Answers 2

1
$\begingroup$

Both solutions work in the sense that, if $Z$ denotes the index of the component of the mixture used to produce $X$, there is equality between the expected weights: $$\mathbb E^X\left[\dfrac{f(X)}{p(X)}\right] = \mathbb E^{X|Z}\left[\dfrac{f(X)}{p(X|Z)}\right] = I$$

Here is a quick illustration of the fact:

#target function
f <- function(x) sin(3*x)
#mixture
p <- function(x) .3*dbeta(x,1,2)+.7*dbeta(x,2,4)
#complete sample
N=1e6
z=sample(0:1,N,replace = TRUE,prob = c(3,7))
y=rbeta(N,shape1 = 1+z,shape2 = 2+2*z)
w1=f(y)
w2=w1/p(y)
w1=w1/dbeta(y,shape1 = 1+z,shape2 = 2+2*z)
#estimators
est1=cumsum(w1)/(1:N)
est2=cumsum(w2)/(1:N)

with both estimators having similar behaviour in that simple example (blue for the unconditional version and orange for the conditional version):

![enter image description here

#plots
plot(log(1:N),est1,ty="l",ylim=c(.3,.8),col="midnightblue",xlab="log(n)",ylab="")
lines(log(1:N),est2,col="orange2",lty=2)
abline(h=integrate(f,0,1)$value,col="tomato",lty=3)
$\endgroup$
1
$\begingroup$

I think you should use following steps

lets we want to generate $m$ samples to use in Monte-Carlo estimator

1) Generate $m$ sample from following distribution (with replacement )

\begin{eqnarray} \begin{array}{c|ccc} k & 1 & 2 & \cdots & n \\ \hline probability & \frac{\lambda_1}{\sum \lambda_i} & \frac{\lambda_2}{\sum \lambda_i} & \cdots & \frac{\lambda_n}{\sum \lambda_i} \\ \end{array} \end{eqnarray}

and save it(in vector named $K$,In standard notation $\sum \lambda_i=1$ So you can just ignore $\sum \lambda_i$).

2) Now for each $K=k$ generate sample from $\mathbb P_k(x)$

lets do it with a simple example

$X\sim .2 Normal(0,1)+.8 Normal(5,1)$

lets we want to estimate $E(X)$ (that exact value is $4$ and estimate should be near $4$). I do it in R software.

  > m<-100000
  > k.vector<-sample(1:2,size=m,p=c(.2,.8)/sum(c(.2,.8)),replace=T)
  > l1<-length(which(k.vector==1))
  > l2<-m-l1
  > random.g<-c(rnorm(l1,0,1),rnorm(l2,5,1))
  > mean(random.g)
  [1] 4.003443

This algorithm is a simple method based on conditional porbability

$$f_X(x)=\sum \mathbb P(X=x|K=k) P(K=k)$$ so in first step we produce one $k$ and then generate one $x$ form $\mathbb P(X|K=k)=\mathbb P_k(x)$. Next use this generated sample to estimate what you want.

$\endgroup$
1
  • $\begingroup$ What I asked was vague so I edited my post. $\endgroup$
    – bitWise
    Apr 9, 2020 at 10:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.