I implemented a metroplis sampler for a 1D gaussian mixture, the target distribution looks like this:
I use a 1D normal distribution as propsal, that is each candidate is sampled from a normal distribution centered around the last accepted sample with a certain standard deviation.
If I use a small variance for the proposal distribution my samples are distributed according to the target distribution although it takes a significant number of samples to achieve a good coverage. If I increase the variance of the propsal distribution my acceptance ratio rapidly drops which was expected, but also my 10^6 samples are not distributed according to the target distribution. Especially the difference between the number of samples generated for each mode of the distribution vanishes: I could see why this happens in the case for a proposal distribution with small variance but I don't really understand the behaviour of the sampler in the case for $\sigma = 10$ or $100$.
Could anyone explain this behaviour to me?
for i=1:n while 1>0 candidate = normrnd(x_curr,sd_prop); p_candidate = eval_mixture_pdf(candidate); if p_candidate > p_curr || rand() < (p_candidate / p_curr) samples(i) = candidate; x_curr = candidate; p_curr = p_candidate; break; end end end