# Log likelihood function for binary classification

I need help with this following task. There is a binary classification problem where each observation xn is belong to one of two classes (t = 0 and t = 1). The training data points are sometimes mislabeled. For every data point xn, instead of having a value t for the class label, we have a value πn representing the probability that tn = 1. Given a probabilistic model p(t = 1φ) what is the log likelihood function?

• If this is homework, or you're working through a course on your own, consider adding the "self-study" tag to this question. – shadowtalker Mar 25 '15 at 12:13
• Yes, it is an exercise from a patter recognition book. – Kate Mar 25 '15 at 13:06
• Pretty much stuck completely. – Kate Mar 26 '15 at 8:20

Equation 12.6 onwards from these lecture notes may be helpful.

The log-likelihood for a probabilistic model for binary classification is

$\sum_{i=1}^n y_i \log p(x_i) + (1 − y_i) \log (1 − p(x_i))$,

where $p(x_i)$ is the model predicted probability that the $i$-th observation is a 1, and $y_i$ is the $i$-th observation for the response.

In summary, sum up the logs of the predicted probabilities where the actual response was one, and add this to the sum of the logs of (1 - the probabilities) whenever the actual response was zero.

• Slight correction to terminology: $p(x_{i})$ is not the probability that the $i$th observation is _classified_ as a 1; it is the probability that the $i$th observation on $Y$ is 1. – Frank Harrell Nov 15 '15 at 23:36

A probabilistic model and its associated likelihood is at odds with classification. If you are doing classification you are typically using a very sub-optimal objective function and not using the likelihood. Likelihoods are for risk prediction (i.e., what logistic regression is for).