# Questions tagged [confusion-matrix]

A confusion matrix is a contingency table used to evaluate the predictive accuracy of a classifier. Confusion matrix is the 2x2 frequency table with counts "True positive", "True negative", "False positive", "False negative", relating classifying to a class of interest vs. else class. But in a broader sense, any frequency kxk crosstabulation "Predicted" x "Actual" classes can be called a confusion matrix, in the context of evaluation of a classifier.

274 questions
Filter by
Sorted by
Tagged with
27 views

### When are we satisfied with the performance of a classifier?

I am training a multi-head classifier (based on Resnet18) for a multilabel classification task. The dataset I am using to train the classifier is noisy and to improve the accuracy for the worse-...
• 137
8 views

### Calculating entropy-based indices based on a frequency cross-tabulation of objects in Python

I was wondering whether there exist a Python solution for calculating Mutual Information, Completeness, V-Measure, etc., based on a pair-confusion matrix. MI defined as H(C_0) + H(C_1) - H(C_O&C_1)...
• 61
37 views

### Are Type I and Type II errors in multiclass problem appear to be the same?

As far as I know from binary classification FP error is a type 1 error FN error is a type 2 error I have this confusion matrix generated: And here I found how to read this confusion matrix: As you ...
• 101
1 vote
23 views

### How to assign costs to the confusion matrix

I am trying to assign costs to the confusion matrix. That is, in my problem, a false positive (FP) does not have the same cost as a false negative (FN), so I want to assign to these cases a cost "...
• 135
62 views

In a machine learning context, I am working on a binary classification problem. There is a source of truth $T$ for labels, and a labeling process $A$ which is not perfect and makes errors compared to $... • 1,387 0 votes 0 answers 27 views ### Confusion matrix vs ROC-AUC curve I'm using the lares packages in combination with H2O and with ... • 2,207 0 votes 0 answers 15 views ### Understand AUCs of different models I'm testing two models against each other: provides an AUC of 93,94 % with TP = 99,9 %, TN = 0 %, FP = 100 % and FN = 0.1 %. provides an AUC of 92,78 % with TP = 98,8 %, a TN = 30,6 %, FP = 69,4 % ... • 2,207 0 votes 0 answers 23 views ### Make ROC graph with my problematic [duplicate] For a personal project, i'm trying to figure out how to trace some ROC/AUC graph with my current problematic. I have a list of thunder flashes, and i'm trying to find if a meteorological variable (... 0 votes 1 answer 21 views ### How to interpret this confusion matrix and roc curve? I got these two images for evaluating a RF: I wonder why the ROC curve seems to be so good while the confusion matrix shows that the True Positive isn't so good with only ~16 %? By looking only at ... • 2,207 0 votes 0 answers 27 views ### Interpreting puzzling result from a confusion matrix from two different algorithms I use a data set from Kaggle on Home Credit Default Risk to learn more about classification algorithms. If I run two very basic algorithms I get puzzling result from the confusion matrix. The logistic ... • 101 1 vote 1 answer 71 views ### Logistic Regression: What is the value for precision when recall (true positive rate) is 0? A quick overview of definitions before I get into the question: True Positive (TP): An actual positive that the model classified as positive False Positive (FP): An actual negative that the model ... • 211 0 votes 0 answers 28 views ### Does a single F-score make sense for multiclass classification? I'm newish to ML and trying to compute some performance metrics for a K-means classifier. I've constructed the following confusion matrix, and computed the per-class precision, recall, and F-scores (... • 143 1 vote 0 answers 38 views ### Confusion matrix error [closed] I have been trying to do support vector machine classification in R and confusion matrix has been showing the error even after changing into factors confusionMatrix(as.factor(yTestPred), as.factor(... • 11 3 votes 1 answer 154 views ### why I am getting TP and FP in confusion matrix is 0, how to get it right? why I am getting TP and FP in confusion matrix is 0, how to get it right? tp = 0,fp = 0,tn = 9847,fn = 18 1 vote 1 answer 23 views ### Estimate sample size from a variable population Context: I want to measure accuracy, precision, and recall for individual raters. Each rater completes a variable amount of labels, for ex. rater A may complete 500 in a given time period while rater ... 1 vote 1 answer 193 views ### Creating confusion matrix and calculating TP, TN,FP, FN I want to generate a confusion matrix for class 2. There is a total of four input classes namely 0,1,2,3. In particular test data batch, there is no input ... • 113 0 votes 2 answers 307 views ### My training accuracy is 1.0 and my test accuracy is 0.994. Am I overfitting for a multiclass classification? This is a multiclass classification for an imbalanced dataset. I set the class_weight for this model to "balanced". I have a perfect training accuracy (1.0) and a nearly perfect testing ... 0 votes 0 answers 176 views ### understanding confusionMatrix output in R My PI wants me to run a confusion matrix on two variables that are 0/1 and should (but probably aren't) reasonably close. I have never run a confusion matrix before and have only a limited idea of ... • 329 1 vote 1 answer 67 views ### How to compare ctree and cforest approaches on the classification task? I doing some numerical experiments with ctree() and cforest() functions from the partkid package. I am using the Wine Quality Data Set. ... • 840 0 votes 1 answer 19 views ### In confusion matrix prediction, has the training data used for prediction been seen by the model I understand that Confusion Matrix evaluates prediction results with metrics such as precision, recall, accuracy and F1-Score. My question is regarding the data used for prediction. Is it okay if this ... 2 votes 1 answer 35 views ### When comparing classifiers on different datasets with different prevalences, is it valid to calculate prevalence-adjusted PPV? Scenario: comparison of 2 different binary classifiers Both classifiers report sensitivity and specificity and number actually positive (P), but classifier 1 is tested on a dataset with prevalence 20%,... 0 votes 1 answer 99 views ### Why does confusion matrix shows inverse output? I am working on a logistic regression binary classification with 1000 rows and 28 columns While I have split my dataset into train and ... • 1,872 0 votes 0 answers 13 views ### Likelihood of True Positive only from TPR and FPR When preparing for an interview, I got the following question (on this medium link), Given a medical test produce a 1% False Positive rate (FPR), and the population True Positive Rate (TPR) is 5%, ... • 103 0 votes 0 answers 140 views ### How to interpret a confusion matrix in a multi-class prediction task? I am struggling to interpret a confusion matrix in a multi-class prediction task. I have the data in three classes: 'A', 'B', 'C', and I used SVC to do the classification. In order to see the ... • 21 0 votes 0 answers 16 views ### Validation and test metric with k-fold CV I am following this procedure from sklearn page: I got a dataset with 42000 samples, and i have used train_test_split to split ... • 101 0 votes 0 answers 96 views ### Can you calculate recall/precision/f1 score for "continuous variables" My goal is to calculate some performance metrics (precision, recall, f1,accuracy) on a binary variable (yes or no) but have dates associated with each data point. Attaching dates to this binary ... • 1 1 vote 0 answers 40 views ### Calculating overall Precision and Recall, one versus all I've got several confusion matrices, all of binary classification (negative, positive). I would like to get general scores of all the matrices combine. Problem is, that the data is not balanced at all.... • 11 1 vote 1 answer 37 views ### How to fully evaluate a multiclass classification problem? When you have a multiclass classification problem, what is the right way to evaluate it's performance? What I usually do is to display the confusion matrix and the ... 0 votes 1 answer 41 views ### How to practically calculate the accuracy of each class in muliclass classification problem? I have the following confusion matrix: ... 0 votes 0 answers 23 views ### Difference in model result when scoring data I am trying to predict/score on an out-of-time data applying the same model I had used to train and predict on test data. In the previous model, the recall and precision for correctly predicting 1s ... 2 votes 2 answers 484 views ### How can accuracy be greater than my precision, recall and F-Score metrics? I have trained two models to detect gestures using ambient light and solar panels. I am now testing the two models in different light scenarios. I have a Convolutional Neural Network model that ... • 21 0 votes 0 answers 44 views ### In multi-class, is the average accuracy of each class in confusion metrics equal to the accuracy calculated from cross validation? When I calculate accuracy through cross validation, it gives me a different accuracy than when I calculate through confusion metrics. Why does it give different accuracy? Is overall calculated ... 1 vote 1 answer 175 views ### Interpreting NaN values for precision in Confusion Matrix Please refer to the confusion matrix here: https://imgur.com/a/Iq1epre Would I get precision values of NaN because of 0/0 in the right most columns? Is that even possible? How should I interpret this? ... • 47 1 vote 1 answer 72 views ### How to obtain conditional use accuracy equality with communities with different real positive rates? The short version is that I would like to know what the confusion matrices (numbers of true positives, false positives, true negatives, and false negatives) should be to achieve conditional use ... 0 votes 0 answers 16 views ### Testing a churn model trained on fix period data A Churn model that needs to predict the customers more likely to churn after 9 months (from the executon time),the model was trained on customer's (retail bank) transactions/ loans 3 years data ... 0 votes 0 answers 28 views ### Is there a F1-score for the negative class? I am analysing some data and calculated the$F1 = \frac{2TP}{2TP + FP + FN}$I'd like to know if there is a name to a negative-class based F-score, something like$F0 = \frac{2TN}{2TN + FP + FN} = 2 \...
40 views

We have 2 Classification models (Random forest on balanced Data Set), the first one classify a Bank's client as a Churner (closing his account) or an active client, the second one classify a Bank's ...
330 views

### How to compute a 'pair confusion matrix'?

I don't really understand how the pair confusion matrix (used for example in comparing of clusterings) is calculated... ...
• 23
83 views

### Should train and test dataset label distribution match reality?

Assume I’m building a binary classification model to detect a disease that according to research is present in 3% of the population. I’m given a balanced dataset to train the model where half of the ...
• 275
58 views

### Interpretation of number of observations in confusion matrix after k-fold cross validation

I completed a 5-fold cross validation and produced a confusion matrix that (I believe) summarizes the results of the 5-fold cv; however, I don't understand why there are only 8 observations in the ...
• 19
1 vote
734 views

### Creating a confusion matrix for every fold from k-fold cross validation in R

I am attempting to run a 5-fold cross validation in R for a logistic regression model followed up by confusion matrices for each "fold." However, my code is only producing one confusion ...
• 19
14 views

### How should I calculate recall and precision on unbalanced data?

I have a model that classifies people into three segments (X, Y, Z). The model says that there are 150 of X, 250 of Y, and 5000 of Z. However, I don't know for certain that is the overall prevalence ...
• 156
140 views

### Why is it called Sensitivity/Recall and Specificity?

Where do the terms: Sensitivity, Recall and Specificity come from historically? I've been looking for an answer for quite some time but to no avail. I understand the formulae and what they mean but I ...
• 139
15 views

### How do I analyse my confusion matrix?

How do I analyze my confusion matrix outcome in terms of FP/TP/FN/TN, precision, etc? ...
• 11
11 views

### Training and Testing accuracy High, but actual testing low

I have approximately 377 samples of an audio event A and the exact amount of another event B. I am using a 10 fold cross validation method to classify the binary event. My Training accuracy comes in ...
• 125
214 views

### What is no information rate in confusionMatrix in r? [duplicate]

I wrote a logistic regression and used confusionMatrix function of caret library to check the accuracy of this model. Here is my code. ...
139 views

### Multi-class optimal threshold selection

What are the methods typically used for optimal threshold selection of multiclass classification problem ? I implemented the detector/classifier and generated the confusion matrix, but would like to ...
• 121
31 views

### Why on a Confusion Matrix are Independent and Dependent Axis Transposed?

Normally, a graph's X axis contains the independent values and the Y axis contains the dependent values. However, on a Confusion Matrix, the X is dependent (predicted) and Y is independent (true). Why ...
• 103