Log likelihood function for binary classification

Question

I need help with this following task. There is a binary classification problem where each observation xn is belong to one of two classes (t = 0 and t = 1). The training data points are sometimes mislabeled. For every data point xn, instead of having a value t for the class label, we have a value πn representing the probability that tn = 1. Given a probabilistic model p(t = 1φ) what is the log likelihood function?

Answer 1

Equation 12.6 onwards from these lecture notes may be helpful.

The log-likelihood for a probabilistic model for binary classification is

$\sum_{i=1}^n y_i \log p(x_i) + (1 − y_i) \log (1 − p(x_i))$,

where $p(x_i)$ is the model predicted probability that the $i$-th observation is a 1, and $y_i$ is the $i$-th observation for the response.

In summary, sum up the logs of the predicted probabilities where the actual response was one, and add this to the sum of the logs of (1 - the probabilities) whenever the actual response was zero.

Answer 2

A probabilistic model and its associated likelihood is at odds with classification. If you are doing classification you are typically using a very sub-optimal objective function and not using the likelihood. Likelihoods are for risk prediction (i.e., what logistic regression is for).