3
$\begingroup$

I was wondering if there was a simple solution to get recall and precision value for the classes of my classifier?

To put some context, I implemented a 20 classes CNN classifier using Tensorflow with the help of Denny Britz code : https://github.com/dennybritz/cnn-text-classification-tf .

As you can see at the end of text_cnn.py he implements a simple function to compute the global accuracy :

# Accuracy
        with tf.name_scope("accuracy"):
            correct_predictions = tf.equal(self.predictions, tf.argmax(self.input_y, 1))
            self.accuracy = tf.reduce_mean(tf.cast(correct_predictions, "float"), name="accuracy")

Any ideas on how i could do something similar to get the recall and precision value for the differents categories?

Maybe my question will sound dumb but I'm a bit lost with this to be honest.

$\endgroup$

closed as off-topic by gung, kjetil b halvorsen, Peter Flom Jun 23 '17 at 11:48

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – gung, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.

2
$\begingroup$

Ok so i solved my problem, just posting my solution here in case somebody has the same issue.

Basically what I did is construct "by hand" a confusion matrix which is a 2D list of 20 rows/20 columns (20 because I had 20 categories). I filled this matrix at every step of the training by comparing the predicted category and the labeled category.

Example when predicted category is number 16 and the labeled category is 7:

confusion_matrix[16][7]+=1

This confusion matrix allowed me to compute recall and precision values in the end by using the classic formula you can see here : https://en.wikipedia.org/wiki/Precision_and_recall

$\endgroup$
0
$\begingroup$

I'm not an expert in tensorflow, but you can obtain the confusion matrix as

y_ = tf.placeholder(tf.float32, shape=[None, 2])
y = last_layer(d/2,2,h_fc1_drop)
confusion_matrix_tf = tf.confusion_matrix(tf.argmax(y, 1), tf.argmax(y_, 1))

Then after training you can obtain the CM as

cm = confusion_matrix_tf.eval(feed_dict={x: X_train, y_: y_train, keep_prob: 1.0})

Prediction and recall can be derived from cm using the typical formulas. For those interested I have created these 3 helpers:

def beautyCM(cm, ind=['True pos', 'True neg'], cols=['Pred pos', 'Pred neg']):
    return pd.DataFrame(cm, index=ind, columns=cols)

def precision(cm):
     # prec = TP / (TP + FP)
    try:
        return round(float(cm.loc['True pos', 'Pred pos']) / \
                     (cm.loc['True pos', 'Pred pos'] + cm.loc['True neg', 'Pred pos']), 4)
    except ZeroDivisionError:
        return 1.0


def recall(cm):
    # prec = TP / (TP + FN)
    try:
        return round(float(cm.loc['True pos', 'Pred pos']) / \
                 (cm.loc['True pos', 'Pred pos'] + cm.loc['True pos', 'Pred neg']), 4)
    except ZeroDivisionError:
        return 1.0

Just copy-pasted, so many variables defined from previous code. Let me know if there is any problem

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.