I am a computer scientist, so I have a background at maths (however limited). I am reading about posterior distribution from here http://en.wikipedia.org/wiki/Posterior_distribution .

It says there:

The posterior probability is the probability of the parameters θ given the evidence X: p(θ|x).

It contrasts with the likelihood function, which is the probability of the evidence given the parameters: p(x|θ).

My question is firstly can you provide me with a very simple example to understand better these concepts? And in addition in machine learning, what we want isn't the probability p(Class|x)?

Thanks a lot

  • 2
    $\begingroup$ There are many many examples out there already, which you can easily find by googling. $\endgroup$
    – jerad
    Jan 26 '14 at 20:22
  • $\begingroup$ If you define the 'parameter' $\theta$ as being $P(Class | x)$ then it's clear that that's what you actually want. A fairly complete exposition with an ML slant is Bishop's book and a very short piece that helpfully related posteriors to likelihood functions in a classification context is Jordan 1995 $\endgroup$ Jan 26 '14 at 22:57

Likelihood is something that you design. Using Likelihood and prior you compute posterior and use it for inference.

Watch this for details: http://videolectures.net/mlss09uk_bishop_ibi/

  • 1
    $\begingroup$ Thank you! "Using Likelihood and prior you compute posterior and use it for inference.", was, what I was looking for. $\endgroup$ Feb 15 at 12:31

Not the answer you're looking for? Browse other questions tagged or ask your own question.