3
$\begingroup$

I'm playing around with a Metropolis-Hastings MCMC algorithm as described in this post.

I made an example data set with points taken from two lines shown below.

The dataset I made

Both lines have a y-intercept of 0 and the normally distributed noise has standard deviation of 0.1 for both. One line has a slope of 1 and one line has a slope of 10.

I tried computing the posterior distribution for all three parameters; slope, intercept, and standard deviation of noise. I wanted to see that the slope parameter posterior had a bi-modal distribution, with one mode at 1 and the other at 10. However, it just looked like a normal distribution with a mean around 5.

What is going on? I expected the Bayesian analysis to show that there were two values for the slope that are equally likely. I don't really see how it is more useful than least squares regression otherwise.

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ You did not manage because your example concerns with mixture distribution (example here stats.stackexchange.com/questions/178321/…) and it needs more complicated tools. Also please tell us more about your approach to this problem since it is not clear from your question. $\endgroup$ – Tim♦ Oct 29 '15 at 14:56
  • 2
    $\begingroup$ We wrote a paper on estimating mixtures of regression, published in JCGS, 2003, this could suggest you ways of simulating from the posterior. $\endgroup$ – Xi'an Oct 29 '15 at 15:31
3
$\begingroup$

If you literally used the code from that post you linked to, then it's fairly clear what happened. You shouldn't expect that your posterior will be bimodal. It will be unimodal because of the likelihood which you chose. Doing the Bayesian analysis will essentially give you the same answer as running a linear regression using lm, and you can see that fitting a line through this set of points with least squares will yield the sort of result that you got!

The only way you could get a bimodel posterior is if you used a strongly bimodal prior with peaks around $1$ and $10$. Or alternatively, you should use a different likelihood (which I think is what you want to do.)

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.