I've put together a binary classifier using Keras' Sequential model. Of its errors, it predicts with false negatives more frequently than false positives.
This tool is for medical application, where I'd prefer a false positive as to err on the side of caution.
How might I try to tweak the model to prefer one class over the other?
A standard way to go about this is as follows:
As mentioned in Dave's answer, instead of taking the binary predictions of the Keras classifier, use the scores or logits instead -- i.e. you need to have a confidence value for the positive class, instead of a hard prediction of "1" for the positive class and "0" for the negative class. (most Keras models have a model.predict() method that gives you the confidence for each class).
Now plot a ROC curve, for which sklearn has some nifty functions ready-made: https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html. This curve basically plots the true positive rate versus the false positive rate, which are obtained by setting various thresholds on the predicted confidence and calculating the true positive rate (TPR) and false positive rate (FPR).
Looking at the ROC curve, you can select a point you would prefer (i.e. with very few false negatives and an acceptable number of false positives). The threshold that gives this (TPR, FPR) point should be the operating point of your classifier (i.e. apply this threshold on the model confidence for "class 1").
Remember that your neural network outputs probabilities, not classifications. It is up to you to use the probabilities to make classifications. The software default is to use a threshold of $0.5$ to determine classification, but that might not be the right number for you.
While there are two outcomes, there is likely a third option in the decision-making process of your medical application: “inconclusive, collect more information.” Perhaps you will want something like below $0.3$ is category $0$, above $0.7$ is category $1$, and between $0.3$ and $0.7$ is inconclusive.
This idea of neural networks outputting probabilities instead of categories is related to what are called proper scoring rules and (better yet) strictly proper scoring rules. Among others on here, Frank Harrell and Stephan Kolassa are fans, the former of whom writes about them on his blog.
https://www.fharrell.com/post/class-damage/
https://www.fharrell.com/post/classification/
Also of interest: Proper scoring rule when there is a decision to make (e.g. spam vs ham email)
c is the cost of a false-positive and (1-c) the cost of a false negative, and you have a perfectly calibrated model, then minimal cost is achieved at a probability cutoff equal to c. Conversely, any choice of a cutoff also is making a (perhaps unconscious) choice of relative costs. See this answer for more details and links.
$\endgroup$
Here are a couple of options to bias the network towards the class you want:
1- Modify the class weights. For example, in order for your algorithm to treat every instance of class 0 as 5 instances of class 1, you can do:
class_weight = {0: 5., 1: 1.}
model.fit(X, y, class_weight=class_weight)
2- Modifying the classification threshold based on the ROC curve, as pointed out by Dave and AruniRC.
3- Oversampling the training samples from the class that you want to prioritize, e.g.:
X_neg = X[y == 0]
X_pos = X[y == 1]
ids = np.arange(len(X_neg))
choices = np.random.choice(ids, len(X_pos) * 5)
X_neg = X_neg[choices]
X_pos = X_pos
y_neg = np.zeros(len(X_neg), dtype=np.int8)
y_pos = np.ones(len(X_pos), dtype=np.int8)
X = np.vstack([X_neg, X_pos])
y = np.stack([y_neg, y_pos])
The problem you stated is generally a common problem in the case of imbalanced datasets, where you might have many more samples in one class compared to another (which commonly happens in medical datasets). Most of these points were taken from the Keras tutorial for handling imbalanced datasets.
