Logistic Regression: What is the value for precision when recall (true positive rate) is 0?

Question

A quick overview of definitions before I get into the question:

• True Positive (TP): An actual positive that the model classified as positive

• False Positive (FP): An actual negative that the model classified as positive

• False Negative (FN): An actual positive that the model classified as negative

• Recall (True Positive Rate, TPR): $\frac{TP}{TP + FN}$

• Precision: $\frac{TP}{TP + FP}$

• Logistic Threshold: A probability above which a sample is classified as positive and below which is classified as negative. It is the grey line in the figure:

• Confusion Matrix: Summary of classification results

• Steps to generating Precision-Recall curve: 1.) Choose a threshold. 2.) Generate confusion matrix. 3.) Calculate the TPR and Precision from confusion matrix and plot the point

• Example of Precision-Recall curve:

We can see from this example that the precision is 1 when the recall is 0. I find this confusing...

---

My question:

Let's say we choose a threshold of 1. Thus, all the samples will be classified as negatives. Thus there will be no true positives, no false positives, and likely a bunch of false negatives.

The calculated recall (TPR) would be: $\frac{0}{0 + FN} = 0$

The calculated precision would be: $\frac{0}{0 + 0} = \frac{0}{0}$

So how come the precision-recall curves I see have precision = 1 when recall (TPR) = 0?

Answer 1

You can't divide by zero, so precision is undefined. It's a pathological case, so it doesn't return meaningful results.

Answer 2

	Predicted Positive (PP)	Predicted Negative (PN)
Real Positive (RP)	True Positive (TP)	False Negative (FN)
Real Negative (RN)	False Positive (FP)	True Negative (TN)

We can think on Recall as the conditional probability of your model returning a true positive $(TP)$ conditional on the real value being a real positive $RP = (TP + FN)$.

$$ recall = P(TP|RP) = \frac{TP}{TP+FN} $$

In the same way, we can think on Precision as the conditional probability of your model correctly predicting a True positive $TP$ conditional on the prediction being positive $PP = (TP + FP)$

$$ precision = P(TP|PP) = \frac{TP}{TP+FP} $$

From equation of recall we can observe that, as threshold approximates 0, the probability of estimating a True Positive $TP$ given that the real value is Real Positive $RP$ is 1. That is true always, as you will never have False Negatives $FN$ if you never give a Negative answer.

From the second equation, on precision, it can be observed that there is something fishy. Since we sat our threshold to 1, How can we Find the probability of a True positive ($TP$) given that the Predicted value is positive ($PP$) if by definition of the Threshold there is no Positive Predicted value $PP$ at all??? This is where your answer gets undefined.

Extra:

The logistic regression models the change in probabilities of a Variable $(Y)$ given another Variables $(X)$. Usually, we usually model a logistic function (sigmoid function) as:

$ ln(\frac{Y}{1-Y}) = F(X) + \epsilon$

and one may minimize the sum of square errors; epsilon $\epsilon$. You can model F as you want, but usually is just a linear function on $X$.

Moving the threshold is actually a common practice, but we have to understand the tradeoffs. The midpoint 0.5 is where your model is equally likely to assign the positive or negative class.

However, we can push it higher so that we get only True positive $TP$ when we have more evidence about the value being likely positive (For example, when giving a credit, you may want to be certain of being paid back) In such a case you would get a better precision score, more True Positives per False Positives. But predict more False negatives, and as such, lower Recall.

In the same sense, we can push the threshold lower if we want to minimize the probability of getting a false negative, in such a case we would be accepting more values, even when we do not have sufficient information about them being Positive. This is the case you could apply for example in covid, is better to isolate a person for 15 days, even if incorrectly isolated ($FN$), than exposing the person to the society and get higher infection rates. Higher recall, lower precision.

When you take it to the limit, then why do you even model it? Like, if you will send everyone to house in covid, regardless of them being sick or not, why even testing them (set threshold = 0)? If you will not give anyone a credit, why even calculate the probability of them paying back (set threshold = 1)?