I've put together a binary classifier using Keras' Sequential model. Of its errors, it predicts with false negatives more frequently than false positives.

This tool is for medical application, where I'd prefer a false positive as to err on the side of caution.

How might I try to tweak the model to prefer one class over the other?

  • 3
    $\begingroup$ You’d better talk to someone who you hope will use the classifier. Assuming a false positive is preferable to a false negative purely because the application is medical could be a major error. There’s plenty of medical situations where a false negative may be preferable, or even designed for. $\endgroup$
    – rhialto
    Commented Jan 24, 2021 at 18:20
  • $\begingroup$ @rhialto, Yes, the classifier is meant to draw attention to problems areas of a medical measurement for further human review. In my situation I'd rather put human eyes on an inconclusive measurement than throw it out. $\endgroup$ Commented Jan 24, 2021 at 18:23
  • 2
    $\begingroup$ so you have a specific situation in mind. Is it you who wants human to review, or have you been told that’s how it should work by a regulator or physician? Because the human might not want to look at a load of false positives. $\endgroup$
    – rhialto
    Commented Jan 24, 2021 at 18:26
  • $\begingroup$ Good question. I was told by a clinician how they'd prefer to use a classifier in their work, but we have not spoken about the specifics of false positives/false negative bias. $\endgroup$ Commented Jan 24, 2021 at 18:28
  • 1
    $\begingroup$ My model is validated at 97% accuracy (3 false negative classifications out of a hundred total classifications in the validation set. Original data is balanced for both classes, almost 50/50). So they wouldn't get a slew of false positives as far as I can tell (and if they did it'd be necessary information for me to further update my model) $\endgroup$ Commented Jan 24, 2021 at 18:31

3 Answers 3


A standard way to go about this is as follows:

  1. As mentioned in Dave's answer, instead of taking the binary predictions of the Keras classifier, use the scores or logits instead -- i.e. you need to have a confidence value for the positive class, instead of a hard prediction of "1" for the positive class and "0" for the negative class. (most Keras models have a model.predict() method that gives you the confidence for each class).

  2. Now plot a ROC curve, for which sklearn has some nifty functions ready-made: https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html. This curve basically plots the true positive rate versus the false positive rate, which are obtained by setting various thresholds on the predicted confidence and calculating the true positive rate (TPR) and false positive rate (FPR).

  3. Looking at the ROC curve, you can select a point you would prefer (i.e. with very few false negatives and an acceptable number of false positives). The threshold that gives this (TPR, FPR) point should be the operating point of your classifier (i.e. apply this threshold on the model confidence for "class 1").


Remember that your neural network outputs probabilities, not classifications. It is up to you to use the probabilities to make classifications. The software default is to use a threshold of $0.5$ to determine classification, but that might not be the right number for you.

While there are two outcomes, there is likely a third option in the decision-making process of your medical application: “inconclusive, collect more information.” Perhaps you will want something like below $0.3$ is category $0$, above $0.7$ is category $1$, and between $0.3$ and $0.7$ is inconclusive.

This idea of neural networks outputting probabilities instead of categories is related to what are called proper scoring rules and (better yet) strictly proper scoring rules. Among others on here, Frank Harrell and Stephan Kolassa are fans, the former of whom writes about them on his blog.



Also of interest: Proper scoring rule when there is a decision to make (e.g. spam vs ham email)

  • 8
    $\begingroup$ In particular (+1), the probability-value cutoff is simply related to the relative cost of false-positive and false-negative classifications. If costs are scaled such that c is the cost of a false-positive and (1-c) the cost of a false negative, and you have a perfectly calibrated model, then minimal cost is achieved at a probability cutoff equal to c. Conversely, any choice of a cutoff also is making a (perhaps unconscious) choice of relative costs. See this answer for more details and links. $\endgroup$
    – EdM
    Commented Jan 24, 2021 at 15:48
  • 5
    $\begingroup$ Also, in my experience MDs are used to working with scores (a superset of the predicted probabilites). Over here, the right + responsibility to form a diagnosis from dignostic information (including OP's model's predictions) anyways rests with the MD. With few exceptions I'd predict probability at least alongside the labels. (The few exceptions would be pronounced very clearly by the MD, and would in any case require additional information like the different misclassification costs and the relative frequencies of the classes for the precise application - which are for sure not 50:50) $\endgroup$
    – cbeleites
    Commented Jan 24, 2021 at 20:45

Here are a couple of options to bias the network towards the class you want:

1- Modify the class weights. For example, in order for your algorithm to treat every instance of class 0 as 5 instances of class 1, you can do:

class_weight = {0: 5., 1: 1.}
model.fit(X, y, class_weight=class_weight)

2- Modifying the classification threshold based on the ROC curve, as pointed out by Dave and AruniRC.

3- Oversampling the training samples from the class that you want to prioritize, e.g.:

X_neg = X[y == 0]
X_pos = X[y == 1]

ids = np.arange(len(X_neg))
choices = np.random.choice(ids, len(X_pos) * 5)

X_neg = X_neg[choices]
X_pos = X_pos
y_neg = np.zeros(len(X_neg), dtype=np.int8)
y_pos = np.ones(len(X_pos), dtype=np.int8)

X = np.vstack([X_neg, X_pos])
y = np.stack([y_neg, y_pos])

The problem you stated is generally a common problem in the case of imbalanced datasets, where you might have many more samples in one class compared to another (which commonly happens in medical datasets). Most of these points were taken from the Keras tutorial for handling imbalanced datasets.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.