# Log likelihood function for binary classification

I need help with this following task. There is a binary classification problem where each observation xn is belong to one of two classes (t = 0 and t = 1). The training data points are sometimes mislabeled. For every data point xn, instead of having a value t for the class label, we have a value πn representing the probability that tn = 1. Given a probabilistic model p(t = 1φ) what is the log likelihood function?

• If this is homework, or you're working through a course on your own, consider adding the "self-study" tag to this question. Mar 25, 2015 at 12:13
• Yes, it is an exercise from a patter recognition book.
– Kate
Mar 25, 2015 at 13:06
• Pretty much stuck completely.
– Kate
Mar 26, 2015 at 8:20
• @Kate, have you got any progress with this problem? Oct 31, 2021 at 23:17

Equation 12.6 onwards from these lecture notes may be helpful.

The log-likelihood for a probabilistic model for binary classification is

$\sum_{i=1}^n y_i \log p(x_i) + (1 − y_i) \log (1 − p(x_i))$,

where $p(x_i)$ is the model predicted probability that the $i$-th observation is a 1, and $y_i$ is the $i$-th observation for the response.

In summary, sum up the logs of the predicted probabilities where the actual response was one, and add this to the sum of the logs of (1 - the probabilities) whenever the actual response was zero.

• Slight correction to terminology: $p(x_{i})$ is not the probability that the $i$th observation is _classified_ as a 1; it is the probability that the $i$th observation on $Y$ is 1. Nov 15, 2015 at 23:36

A probabilistic model and its associated likelihood is at odds with classification. If you are doing classification you are typically using a very sub-optimal objective function and not using the likelihood. Likelihoods are for risk prediction (i.e., what logistic regression is for).