0
$\begingroup$

I have set up a neural network which has a single output with a sigmoid activation function, which I understand by default is used as a binary classifier where values over 0.5 should belong to class 1 else class 0. After looking at the results of training, it would be a better balance of precision/recall for my task if I set the classification threshold at a lower number, say 0.25.

Is there a proper way to rescale around this new threshold to give a probability of being in a certain class? So for values close to 0.25, its actually around 50% probability of belonging to class 1.

from keras.models import Sequential
from keras.layers import Dense
model = Sequential()
model.add(Dense(1, activation='sigmoid'))
...layers layers layers
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy',precision,recall,f1])
hist = model.fit(X_train.values,y_train.values, epochs=50, batch_size=64,
          verbose=1, validation_data=(X_val.values,y_val.values),
            callbacks=callbacks_list, shuffle=True)

model.predict(X_test) # ... want to use 0.25 as the cutoff threshold
# but also want the probability of belonging to class 1
$\endgroup$
  • $\begingroup$ The output of the network should be the value returned by the sigmoid function, which is used in the loss function directly (typically binary cross entropy). So, it should be pretty easy to lower the threshold as you please. Which library are you using? $\endgroup$ – AndreaL Oct 25 '17 at 14:46
  • $\begingroup$ Keras. Edited to include code snippet. $\endgroup$ – user4446237 Oct 25 '17 at 15:13
3
$\begingroup$

model.predict will output a matrix in which each row is the probability of that input to be in class 1.

If you print it, it should look like this:

[[ 0.7310586 ]
 [ 0.26896983]]

You just need to loop through those values.

for i, predicted in enumerate(predictions):
    if predicted[0] > 0.25:
        print "bigger than 0.25"
        #assign i to class 1
    else:
        print "smaller than 0.25"
        #assign i to class 0

EDIT: It might be worth to play with the weight of the classes. If you weight the 1 class 3 times more, you might get something close to what you want, in a more elegant way.

Here is an example.

$\endgroup$
  • $\begingroup$ I'm interested in the probabilities that they belong to each class given I use a different threshold. Is it valid to just do a linear interpolation and call them probabilities? $\endgroup$ – user4446237 Oct 25 '17 at 16:45
  • $\begingroup$ OK, I would use a logistic function with mid-point 0.25 to do that. Also, you can play with the steepness of the curve (or just leave it to 1). $\endgroup$ – AndreaL Oct 25 '17 at 16:57
  • $\begingroup$ It looks like the logistic function is what I am looking for, if you edit to include I will accept this answer. Is this common practice to tack that on at the end of a NN, which already has a sigmoid output? $\endgroup$ – user4446237 Oct 30 '17 at 18:21
  • $\begingroup$ On second thought, logistic function recentered the 0.25 to 0.5, and capped the output to 1, but given the min value that can come is 0, this gives a different min output. Is there a way to have the 0 input become a 0 output as well? $\endgroup$ – user4446237 Oct 30 '17 at 18:38
  • $\begingroup$ I would increase the steepness of the logistic function to control that. In principle you can also have two different values for the steepness, if the input is above or below 0.25. $\endgroup$ – AndreaL Oct 30 '17 at 23:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.