# Questions tagged [confusion-matrix]

A confusion matrix is a contingency table used to evaluate the predictive accuracy of a classifier. Confusion matrix is the 2x2 frequency table with counts "True positive", "True negative", "False positive", "False negative", relating classifying to a class of interest vs. else class. But in a broader sense, any frequency kxk crosstabulation "Predicted" x "Actual" classes can be called a confusion matrix, in the context of evaluation of a classifier.

164 questions
Filter by
Sorted by
Tagged with
18 views

### Missing name in confusion matrix?

In a confusion matrix, what's the name of the percentage of cases I predict as positive out of the total population? I am in the position of having to use this metric for my project, but I can't find ...
7 views

### Error distribution

I am working on a urgent message classification problem using CRF(Conditional Random Fields). I have obtained a confusion matrix from the model and now I want to check the error distribution i.e. ...
16 views

### Training and predicting a Decision Tree over the same dataset

I wanted to train and predict a Decision Tree over the same dataset because I supposed the metrics will be perfect (overfitting). So I took an imbalanced dataset which I wanted to use as training ...
18 views

### How to back out individual confusion matrix values from scalar measures like sensitivity and specificity?

Consider a standard 2X2 Binary Classification Matrix: TP | FP FN | TN From which we can derive sensitivity and specificity, and other measures. Now, let's assume we have ONLY output measures: ...
7 views

### Measure for in equality of prediction quality in mutlclass classification

I have got a balanced dataset of 10 different classes $y\in\{0,\dots 9\}$. After fitting a ML model I get classification results $y^*$. Despite the data being balanced, the results are not. I want a ...
27 views

### Matrix of confusion - percentage per row or column?

When you build a prediction model on a binary variable, you have a confusion matrix which compares your predictions according to real values. ...
54 views

### Matthews/Phi Coefficient on a 2x2 Contingency Table with Rare Positives

Suppose I want to measure the degree people like gold versus silver. For the sake of argument, let's say I have a contingency table like so: ...
379 views

### Is there a name for this metric: TN / (TN + FN)?

Given a confusion matrix, there's all kind of metrics: Accuracy, Precision, Recall/Sensitivity, Specificity. But I haven't seen any name for the ratio between the TN (True Negative) and the sum of ...
77 views

### Precision, Recall, F1 Score and multiclass confusion matrix

My goal is to evaluate some clustering algorithms. Lets say that I’ve got one set o elements: { 1 ; 2; 3; 4; 5; 6; 7; 8 }. This set was grouped by human into two groups: (1 ; 2; 3; 4; 5; 6; 8) and (7)....
26 views

### Understanding ratio between precision and recall

I have a naive question regarding the ratio of precision and recall. When I build the model, I am able to get precision and recall. Later, I could use this model to make perditions, which is ...
74 views

...
54 views

### Precision-Recall curve interpretation

When given an example confusion matrix: TP = 5000 FP = 1000 FN = 0 ...
266 views

### Find true negatives in a confusion matrix

I'm trying to find the True negative in a confusion matrix, I have computed successfully from scratch the precision and recall/sensibility, now i need to compute the accuracy and specificity. This is ...
83 views

### Is there a name for “normalized accuracy” as a statistic?

In short: I'm using a statistic representing the "normalized accuracy" of a confusion matrix. Is there a formal term for this? $$\text{normacc} = \frac{\text{acc}-\text{thacc}}{1-\text{thacc}}$$ ...
20 views

### How to understand 2X2 confusion matrix one-r?

with this data set when applying one-r with weka choose age group: but I do not understand how weka made this confusion matrix
66 views

### Is it possible to estimate accuracy, precision and recall with the given data?

Background: I talked to my friend today and according to herm(him/her) I can calculate precision, recall and accuracy with the current information. Total instances T: 19,532. Instances belonging to ...
274 views

### Compute standard deviation of accuracy

edit - more information about what the code given should represent The following pseudocode outlines the problem as I have it ...
33 views

### Is it appropriate to use a confusion matrix for frequency data?

I am testing the accuracy of a machine learning approach that counts cars in images. I have both a predicted dataset and a "real" dataset that was generated by a human. For example, this is what my ...
150 views

### Signs of Overfitting in Precision/Recall Curve

plz look at the following figures. As you cann see the precision is always 100% no matter which threshold (x-axis in logarithmic scale) you set! Also the second figure shows that we have a perfect ...
82 views

### Mutual Information from confusion matrix with conditional marginals

I have a table that looks like this ...
538 views

### R - confusionMatrix function in R [duplicate]

I am working on a model in r and below is the output of confusionMatrix I got- ...
32 views

### Multiclass classifier with undefined prediction, how to calculate metrices

I build a multiclass classifier. I want the classifier to predict a few samples with little false positives, rather then many samples with lots of false positives. Therefore I want to choose a ...
31 views

### How to choose metrics for evaluating classification results?

Recently we have developed a python library named PyCM specialized for analyzing multi-class confusion matrices. A parameter recommender system has been added in version 1.9 of this module in order ...
227 views

### How to interpret/deal with low precision confusion matrix?

I am developing a fraud detection model with unbalanced data set , I used Random Forests Algorithm. The confusion matrix showed that model has good sensitivity(recall) but low precision(Pos Pred ...
27 views

### Why is information gain rarely used as a reported effect size?

Commonly, in reporting effect sizes in medical articles, papers typically report: risk difference (RD), relative risk (RR) and odds ratio (OR) (and for variable timescale cases and other situations, ...
58 views

### Display Confusion Matrix properly -

I am trying to print the confusion matrix, but it is getting wrapped after few columns (or characters). I have tried several settings but didn't help: ...
31 views

### how to consider some miss classifications “half correct” in categorical_crossentropy - for a trading system

I have a trading system where the model receives 9 time-series and predict : A - strong down B - week down C - neutral D - week up E - strong up (these classes ...
33 views

### Shuffle vs Non-Shuffle - Confusion matrix reacts differently [closed]

Here is the config of my model : ...
106 views

### Can I compute an F1 score when the test data has no examples of one class?

I am working on a 3-class classification problem. We are cross-validating via a Leave-One-Out Approach, and there are some instances where the test data has no instances of one of my three classes. ...
945 views

### How to find cutoff point in Logistics Regression using R

I have run a Logistics Regression model in my data set. Below is the code: ...
29 views

### Optimal threshold for rejecting in classification model

Let's say I have a model to detect fake product. The model predict the probability whether the product is fake, with 0.0 being authentic, 1.0 is fake. Each true prediction (that product is not fake), ...
431 views

### Calculation of accuracy (and Cohen's kappa) using sensitivity, specificity, positive and negative predictive values

I read How to calculate specificity from accuracy and sensitivity, but I have two diagnostic performance measures more. Please correct me if I am wrong: if Sensitivity=TP/(TP+FN) Specificity=TN/(TN+...
78 views

### Estimating truth and confusion matrix from noisy observations with Expectation Maximization?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: ...
239 views

### Using An ROC Curve to Evaluate a model

I have a number of questions on the ROC curves when being used to evaluate a model. My understanding of them is they can be used to determine the probability cutoff when classifying a row in a dataset ...
16 views

### Trying to read confusion matrices

I am given these 2 confusion matrix for a homework assignment but I am unsure how to interpret the data. I am asked to make a decision with these two matrices. For the first matrix am I interpreting ...
81 views

### Can you compute errors bars from Precision and Recall?

I am performing an object detection task for counting cars in an image. I have the confusion matrix (TP, FP, FP, TN) of the model. I guess TN is just zero in this case, as we aren't detecting where a ...
356 views

### If the position of 0 1 in confusion matrix changes does the formula for sensitivity changes

It may sounds very silly question, but I want to clear myself In confusion matrix we normally takes values like below: ...
54 views

### Rare event confusion matrix

Right now I'm gathering data collected from several individuals and manually classifying 100 randomly selected data points from each one. The condition I'm looking for is pretty rare. It occurs only ...
355 views

### How to determine whether a classifier is significantly better than random guessing?

I have a classifier Y that selects between three categories: A, B and C. I need to be able to quantitatively prove that my model is better (and by how much?), than a random classifier R that randomly ...
2k views

### Threshold to build confusion matrix?

I a have data set with 10 sections of data and each section shows one day observation. I designed the training and test set as follows: 8 sections for training the data and the last two sections for ...
281 views

### Can F1-Score be higher than accuracy?

I'm using sklearn's confusion_matrix and classification_report methods to compute the ...
2k views

### Confidence interval of precision / recall and F1 score

To summarise the predictive power of a classifier for end users, I'm using some metrics. However, as the users input data themselves, the amount of data and class distribution varies a lot. So to ...
215 views

### How to interpret that my model gives no negative class prediction on test set?

I am making multivariate time series classification with TUH seizure corpus dataset I have built this model with Keras, using LSTM layers : ...
3k views

### Is there any difference between Sensitivity and Recall?

In most of the places, I have found that sensitivity=recall. In terms of the Confusion Matrix, the formula for both of these is the same: $TP/(TP+FN)$. Is there any difference between these two ...
83 views

### Confidence interval for a macro/micro average

I'm working on a supervised multi class classifier that labels texts according to three possible classes. I calculated the one-vs-all precision, recall and F1 score for each class and the macro ...
221 views

### Is it possible to get high sensitivity but low precision

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. This dataset is imbalanced with a majority examples belonging to normal class (0) and minority being class labeled ...
309 views

### Educational purpose :Calculation of sensitivity and specificity from confusion matrix for this example

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. The data labelled as 0 denote normal operating conditions and data labelled as ...
2k views

### Lack of understanding of LOOCV

I am trying to utilize LOOCV in the data partition in R. The idea of LOOCV is to train the model on n-1 set and test the model on the only remaining one set. Then, is to repeat this process n times ...