I am getting slightly confused by all the probabilistic classifiers.

  1. The bayes optimal classifier is given as $ max (p(x|C)p(C)) $ and if all classes have equal prior then it reduces to $ max (p(x|C)) $

  2. The likelihood ratio is given as $ \frac{p(x|C1)}{p(x|C2)}$

If I only have 2 classes with equal prior then what is the difference between the bayes optimal classifier and the likelihood ratio ? Wont they both return me the same class as the output ?

  • $\begingroup$ They are totally different things, so could you clarify what makes you consider them "essentially the same"? $\endgroup$
    – Tim
    Commented May 24, 2016 at 19:21
  • $\begingroup$ Sorry I have edited my question. I hope my question is clearer now. $\endgroup$
    – RuiQi
    Commented May 25, 2016 at 1:37
  • $\begingroup$ What you describe seems to be a Bayes classifier, not a Bayes optimal classifier. $\endgroup$ Commented Feb 17, 2019 at 10:47

1 Answer 1


They are not the same, but in you case they could be used for the same purpose.

Optimal Bayes classifier is

$$ \DeclareMathOperator*{\argmax}{arg\,max} \argmax_{c \in C} p(c|X) $$

i.e., among all hypotheses, take the $c$ that maximizes the posterior probability. You use Bayes theorem

$$ \underbrace{p(c|X)}_{\text{posterior}} \propto \underbrace{p(X|c)}_{\text{likelihood}} \underbrace{p(c)}_{\text{prior}} $$

but since using uniform prior (all $c$ are equally likely, so $p(c) \propto 1$) it reduces to the likelihood function

$$ p(c|X) \propto p(X|c) $$

The difference between maximizing the likelihood function and comparing the likelihood ratios, is that with likelihood ratio you compare only two likelihoods, while in maximizing the likelihood you may consider multiple hypothesis. So if you have only two hypotheses, then they will do essentially the same thing. However imagine that you had multiple classes, in such case comparing each of them with all the others pair by pair would be a really inefficient way to go.

Notice that likelihood ratio serves also other purpose than finding which of the two models has greater likelihood. Likelihood ratio can be used for hypothesis testing and it tells you how much more (or less) likely is is one of the models comparing to the other. Moreover, you can do the same when comparing the posterior distributions by using Bayes factor in similar fashion.

  • $\begingroup$ Thank you ! I was planning to edit my question to ask about the maximum likelihood estimation since it looks similar to the bayes classifier ! Thanks for clearing my doubt ! $\endgroup$
    – RuiQi
    Commented May 25, 2016 at 6:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.