I'm struggling to grasp some concepts regarding bayesian learning theory. As I understand it one can classify a dataset by finding a posterior distribution and simply selecting a $w$ in $f(x) = w^TX$ that maximises the density of the posterior distribution. I believe that I read somewhere that this gives the same solution as Full Bayesian inference in the case of Gaussian Process regression but not necessarily in general. Why is this the case? Why might this relation not hold true except in the case of GP regression? Thanks.

  • $\begingroup$ What do you mean by "full Bayesian inference"? I guess you mean estimation the posterior distribution, but then MAP = mode of the posterior distribution, so mode never can't be the same as the distribution unless you have a degenerate distribution consisting of a single point... $\endgroup$
    – Tim
    Jan 5 '18 at 21:48
  • $\begingroup$ Hi, this quora answer explains what i mean: quora.com/… $\endgroup$ Jan 5 '18 at 22:13

As your quote says

Full Bayesian inference means that you learn a full posterior distribution ...

MAP is simply mode of the posterior distribution. They can’t be the same since mode of the distribution is not the same as distribution itself. MAP is a point estimate, while full posterior is a distribution estimate.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.