# Questions tagged [confusion-matrix]

A confusion matrix is a contingency table used to evaluate the predictive accuracy of a classifier. Confusion matrix is the 2x2 frequency table with counts "True positive", "True negative", "False positive", "False negative", relating classifying to a class of interest vs. else class. But in a broader sense, any frequency kxk crosstabulation "Predicted" x "Actual" classes can be called a confusion matrix, in the context of evaluation of a classifier.

290 questions
Filter by
Sorted by
Tagged with
18 views

### What should I do when a test specificity is zero?

Upon doing a test that compares the effect of imaging versus regular tissue biopsy for cancer that has a prevalence of 90%, we ...
808 views

### For a confusion matrix, is there a name for FP / (FP + FN)?

For a confusion matrix, there are a variety of useful rates, ratios and indices. But I cannot find the one I care about: FP / (FP + FN) Of course this measure is ...
1 vote
14 views

### Is there an equivalent for Yates' correction for a confusion matrix-derived metrics?

Given the following table of predictions vs. actual states: ...
17 views

### Confusion matrix : Values for TN are negative

I am trying to get the values for True Positives, False negatives, false positives, and true negatives for my confusion matrix when I run the k-nearest algorithm in Matlab. However, when I calculate ...
65 views

### Overfitting models in mlr3

I'm trying to compare multiple learners on my dataset (called "data") in order to predict a target called "lesionResponse", with custom resampling. Since mlr3 package doesn't allow ...
1 vote
50 views

### Customized F1-Score for multi-class classification

Let's consider a multi-class classification problem with 4 classes: 0, 1, 2, and 3 F1-Score 'macro'-averaged is calculated like that: ...
62 views

### Confidence Informed Confusion Matrix (Threshold Free)

I am using the normalized confusion matrix to aid in quantifying the uncertainty in related observations over time. More specifically, I have a classifier that returns the confidence of each class via ...
1 vote
43 views

### Surprising disparity between Confusion matrix values and AUC?

I'm working on getting a read out of a Logistic regression classification model (setup in Python via Scikit-learn's LogisticRegression() wrapped in a OneVsRestClassifier()). I got the confusion matrix ...
53 views

### Can Accuracy be higher than both sensitivity and specificity?

I came across a paper which reported the following results Accuracy Specificity Sensitivity 97.49% 93.6% 94.3% It seems unusual for accuracy to be higher than both sensitivity and specificity. Is ...
6k views

### My machine learning model has precision of 30%. Can this model be useful?

I've encountered an interesting discussion at work on interpretation of precision (confusion matrix) within a machine learning model. The interpretation of precision is where there is a difference of ...
35 views

### Logistic regression model with different predictors giving same Confusion Matrix

I am new to statistics and regression model and still trying to grasp the concepts of models and its accuracy. One of my assignments is to create a logistic regression model in R using a tele company'...
42 views

### Confusion Matrix with a class created from lower probabilities of other classes

Please comment on the following method: Model was trained on data labeled with 4 classes: header, question, answer an other. Predictions were done on a separate test set, nothing new here. Then ...
274 views

### Calculating the Brier or log score from the confusion matrix, or from accuracy, sensitivity, specificity, F1 score etc

Suppose I have a confusion matrix, or alternatively any one or more of accuracy, sensitivity, specificity, recall, F1 score or friends for a binary classification problem. How can I calculate the ...
1 vote
47 views

### How to measure sensitivity, speficity, PPV and NPV from confusion matrix for multiclass classification [duplicate]

I want to calculate sensitivity, specificity, PPV, and NPV from a confusion matrix for multiclass classification. My objective is to learn the basic concept of constructing a confusion matrix for ...
36 views

### Levels error in complexity matrix

I am trying to classify in R with neural network and Gradient Boosting. My target variable is normally low, medium and high. I'm trying to classify using the codes below, but when I want to create a ...
28 views

### How can you adjust the values in a pair-confusion matrix for chance?

In cluster analysis one of the entries of a pair-confusion matrix is the number of data point pairs that are grouped together by both clustering algorithms. How could one adjust this value for chance ...
1 vote
84 views

### Proof that Recall ≥ F1-score

I need to prove mathematically that the value of Recall is always bigger or equal to the F1-score. The equation would be: $2 \times Precision \times \frac{Recall}{(Precision + Recall)} \le Recall$ ...
1 vote
134 views

### How is it possible to get AUC = 0.72, but sensitivity = 0?

I've trained a Gradient Boost classifier and used it to predict a binary target variable. I plot the ROC, here it is, looks nice and good: However, using the same test data, this is the result of a ...
43 views

### Cross-validation and F1 metric

Cross-validation with metrics such as F1 can be implemented in two ways: For each cross-validation split, calculate F1_split on the validation dataset. F1_result = average_by_splits(F1_split) For ...
1 vote
46 views

### Metric to evaluate binary classification model on imbalanced dataset in order to meet percentage limitations

For a university class I'm working on a imbalanced dataset that has ratio of 43:1 Class_0 to Class_1. Class_1 refer to companies that have declared bankruptcy based on feature columns of the dataset. ...
20 views

### Performance assessment based on conditional probability with a reference model

Say I have a reference binary classifier model with its' confusion matrix. Then I construct another model and I can assess the probability they agree (relative confusion matrix). Is there any way to ...
11 views

### Is there a way to determine the size of a condition positive population?

I am currently working on a project where we are using a random forest classifier (SciKit-Learn's specifically) and we are trying to verify that all three versions we have of the classifier are ...
232 views

### Sensitivity vs. specificity vs. recall

Given a binary confusion matrix with true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), what are the formulas for sensitivity, specificity, and recall? I'm ...
1 vote
187 views

### Is there a version of accuracy weighted by prevalence?

I have been learning the basic terminology for how to think about binary tests involving medical tests. The basic terms are here in this table This is the confusion matrix. My issue is the following. ...
504 views

### Can I apply a confusion matrix to classification tasks outside of ML?

I would like to know if it's possible to use a confusion matrix to measure the performance of a classification tool outside the realm of ML or a statistical model. For example, if I had a small script ...
46 views

### Harmonic mean of false positive and false discovery rates (analogous to F1)

F1 is the harmonic mean of recall (aka sensitivity, or true positive rate, TPR) and precision (aka positive predictive value, PPV). $\text{TPR} = \text{Pr(predicted:Pos | Pos)} =$ TP/P (wikipedia ...
412 views

### Poorly calibrated probabilities but good classification in confusion matrix

I have an imbalanced data set. My goal is to balance sensitivity and specificity via the confusion matrix. I used glmnet in r with class weights. The model does well at balancing the sensitivity/...
1 vote
28 views

### Formula versus Non-Formula Interface Categorical Variables train() glmnet

I am comparing the confusion matrix between the formula interface and the non-formula interface using caret's train() for elastic net. I am trying to understand why the two interfaces produces ...
396 views

### Why is "balanced accuracy" an arithmetic mean instead of harmonic?

In the F1 score, the harmonic mean of precision (Positive Predictive Value) and sensitivity/recall (True Positive Rate), I understand that we use the harmonic mean in order to penalize extreme values ...
488 views

### How to analyse results of classification for time series + sliding windows

Here is my context: I have a time series composed of only 1 features. I want to be able to classify between two classes. To get more information out of these data, I am using a sliding time windows. ...
411 views

### How to calculate the accuracy and error rate in confusion matrix?

(0 - negative outcome, 1 - positive outcome) Am I correct? Because I found a different answer. My answer: Accuracy rate: TP+TN/ALL(58+25/58+44+23+25)=83/150 Error rate: FN+FP(23+44)= 67/150 Someone ...
33 views

### When I convert the result of 'predict' function of logistic regression into factor, should the cutoff be same with the that of dependent variable? [duplicate]

I wanna run glm composed of dependent variable named "exposure", and independent variable named "counts" and "distance" respectively. ...
40 views

### A metric for a big/medium/small ML classification

I am working on an ML classification task which is similar to the following: Apples have to be classified to three classes: Big, Medium and Small. I need a metric which I can use to assess the system. ...
35 views

### When are we satisfied with the performance of a classifier?

I am training a multi-head classifier (based on Resnet18) for a multilabel classification task. The dataset I am using to train the classifier is noisy and to improve the accuracy for the worse-...
1 vote
231 views

### Are Type I and Type II errors in multiclass problem appear to be the same?

As far as I know from binary classification FP error is a type 1 error FN error is a type 2 error I have this confusion matrix generated: And here I found how to read this confusion matrix: As you ...
1 vote
78 views

### How to assign costs to the confusion matrix

I am trying to assign costs to the confusion matrix. That is, in my problem, a false positive (FP) does not have the same cost as a false negative (FN), so I want to assign to these cases a cost "...
97 views

### Two confusion matrices

In a machine learning context, I am working on a binary classification problem. There is a source of truth $T$ for labels, and a labeling process $A$ which is not perfect and makes errors compared to \$...
36 views

### Understand AUCs of different models

I'm testing two models against each other: provides an AUC of 93,94 % with TP = 99,9 %, TN = 0 %, FP = 100 % and FN = 0.1 %. provides an AUC of 92,78 % with TP = 98,8 %, a TN = 30,6 %, FP = 69,4 % ...
25 views

### Make ROC graph with my problematic [duplicate]

For a personal project, i'm trying to figure out how to trace some ROC/AUC graph with my current problematic. I have a list of thunder flashes, and i'm trying to find if a meteorological variable (...
1 vote
48 views

### How to interpret this confusion matrix and roc curve?

I got these two images for evaluating a RF: I wonder why the ROC curve seems to be so good while the confusion matrix shows that the True Positive isn't so good with only ~16 %? By looking only at ...
1 vote
752 views

### Logistic Regression: What is the value for precision when recall (true positive rate) is 0?

A quick overview of definitions before I get into the question: True Positive (TP): An actual positive that the model classified as positive False Positive (FP): An actual negative that the model ...
1 vote
266 views

### Confusion matrix error [closed]

I have been trying to do support vector machine classification in R and confusion matrix has been showing the error even after changing into factors confusionMatrix(as.factor(yTestPred), as.factor(...
2k views

### why I am getting TP and FP in confusion matrix is 0, how to get it right?

why I am getting TP and FP in confusion matrix is 0, how to get it right? tp = 0,fp = 0,tn = 9847,fn = 18
1 vote
34 views

### Estimate sample size from a variable population

Context: I want to measure accuracy, precision, and recall for individual raters. Each rater completes a variable amount of labels, for ex. rater A may complete 500 in a given time period while rater ...
1 vote
719 views

### Creating confusion matrix and calculating TP, TN,FP, FN

I want to generate a confusion matrix for class 2. There is a total of four input classes namely 0,1,2,3. In particular test data batch, there is no input ...
1 vote
4k views

### My training accuracy is 1.0 and my test accuracy is 0.994. Am I overfitting for a multiclass classification?

This is a multiclass classification for an imbalanced dataset. I set the class_weight for this model to "balanced". I have a perfect training accuracy (1.0) and a nearly perfect testing ...
1 vote
217 views

### How to compare ctree and cforest approaches on the classification task?

I doing some numerical experiments with ctree() and cforest() functions from the partkid package. I am using the Wine Quality Data Set. ...
105 views

### In confusion matrix prediction, has the training data used for prediction been seen by the model

I understand that Confusion Matrix evaluates prediction results with metrics such as precision, recall, accuracy and F1-Score. My question is regarding the data used for prediction. Is it okay if this ...