# How to do CV in logistic regression when predicton doesn't work?

Having fit a logistic regression, I want to do cross-validation.

How can I do this? The usual method of computing predictions and calculating the accuracy doesn't work ... because ... there's nothing to predict. My logistic regression predicts a probability, which I can't compare to anything, as my data is binary 1/0 values.

You could try and predict 1 if the predicted probability is greater than 0.5 and vice-versa, but that only works for trivial cases: in my case, the probabilities are always less than 0.5, so this method would always predict y = 0.

• You could use the logloss on validation set predictions as your validation metric Commented Oct 1, 2019 at 13:14

Suppose your prediction for a particular instance $$i$$ to be of class 1 is $$\hat{p}_i$$. If the instance in fact is of class 1, the score is $$s_i:=\log\hat{p}_i$$. If it is of class 0, $$s_i:=\log(1-\hat{p}_i)$$. (It's always the log of the predicted probability for the actual class.)
The you just sum over the scores, $$S:=\sum_i s_i$$. A higher total score is better. (Some people use the opposite convention and minimize the score, then they work with negative logs.)