Questions tagged [confusion-matrix]

A confusion matrix is a contingency table used to evaluate the predictive accuracy of a classifier. Confusion matrix is the 2x2 frequency table with counts "True positive", "True negative", "False positive", "False negative", relating classifying to a class of interest vs. else class. But in a broader sense, any frequency kxk crosstabulation "Predicted" x "Actual" classes can be called a confusion matrix, in the context of evaluation of a classifier.

Filter by
Sorted by
Tagged with
0
votes
1answer
18 views

Missing name in confusion matrix?

In a confusion matrix, what's the name of the percentage of cases I predict as positive out of the total population? I am in the position of having to use this metric for my project, but I can't find ...
0
votes
0answers
7 views

Error distribution

I am working on a urgent message classification problem using CRF(Conditional Random Fields). I have obtained a confusion matrix from the model and now I want to check the error distribution i.e. ...
0
votes
0answers
16 views

Training and predicting a Decision Tree over the same dataset

I wanted to train and predict a Decision Tree over the same dataset because I supposed the metrics will be perfect (overfitting). So I took an imbalanced dataset which I wanted to use as training ...
0
votes
0answers
18 views

How to back out individual confusion matrix values from scalar measures like sensitivity and specificity?

Consider a standard 2X2 Binary Classification Matrix: TP | FP FN | TN From which we can derive sensitivity and specificity, and other measures. Now, let's assume we have ONLY output measures: ...
0
votes
0answers
7 views

Measure for in equality of prediction quality in mutlclass classification

I have got a balanced dataset of 10 different classes $y\in\{0,\dots 9\}$. After fitting a ML model I get classification results $y^*$. Despite the data being balanced, the results are not. I want a ...
0
votes
1answer
27 views

Matrix of confusion - percentage per row or column?

When you build a prediction model on a binary variable, you have a confusion matrix which compares your predictions according to real values. ...
0
votes
1answer
54 views

Matthews/Phi Coefficient on a 2x2 Contingency Table with Rare Positives

Suppose I want to measure the degree people like gold versus silver. For the sake of argument, let's say I have a contingency table like so: ...
2
votes
1answer
379 views

Is there a name for this metric: TN / (TN + FN)?

Given a confusion matrix, there's all kind of metrics: Accuracy, Precision, Recall/Sensitivity, Specificity. But I haven't seen any name for the ratio between the TN (True Negative) and the sum of ...
1
vote
0answers
77 views

Precision, Recall, F1 Score and multiclass confusion matrix

My goal is to evaluate some clustering algorithms. Lets say that I’ve got one set o elements: { 1 ; 2; 3; 4; 5; 6; 7; 8 }. This set was grouped by human into two groups: (1 ; 2; 3; 4; 5; 6; 8) and (7)....
0
votes
0answers
26 views

Understanding ratio between precision and recall

I have a naive question regarding the ratio of precision and recall. When I build the model, I am able to get precision and recall. Later, I could use this model to make perditions, which is ...
2
votes
1answer
74 views
2
votes
1answer
54 views

Precision-Recall curve interpretation

When given an example confusion matrix: TP = 5000 FP = 1000 FN = 0 ...
3
votes
1answer
266 views

Find true negatives in a confusion matrix

I'm trying to find the True negative in a confusion matrix, I have computed successfully from scratch the precision and recall/sensibility, now i need to compute the accuracy and specificity. This is ...
0
votes
0answers
83 views

Is there a name for “normalized accuracy” as a statistic?

In short: I'm using a statistic representing the "normalized accuracy" of a confusion matrix. Is there a formal term for this? $$\text{normacc} = \frac{\text{acc}-\text{thacc}}{1-\text{thacc}}$$ ...
0
votes
0answers
20 views

How to understand 2X2 confusion matrix one-r?

with this data set when applying one-r with weka choose age group: but I do not understand how weka made this confusion matrix
0
votes
1answer
66 views

Is it possible to estimate accuracy, precision and recall with the given data?

Background: I talked to my friend today and according to herm(him/her) I can calculate precision, recall and accuracy with the current information. Total instances T: 19,532. Instances belonging to ...
0
votes
2answers
274 views

Compute standard deviation of accuracy

edit - more information about what the code given should represent The following pseudocode outlines the problem as I have it ...
1
vote
1answer
33 views

Is it appropriate to use a confusion matrix for frequency data?

I am testing the accuracy of a machine learning approach that counts cars in images. I have both a predicted dataset and a "real" dataset that was generated by a human. For example, this is what my ...
0
votes
1answer
150 views

Signs of Overfitting in Precision/Recall Curve

plz look at the following figures. As you cann see the precision is always 100% no matter which threshold (x-axis in logarithmic scale) you set! Also the second figure shows that we have a perfect ...
1
vote
1answer
82 views
1
vote
1answer
538 views

R - confusionMatrix function in R [duplicate]

I am working on a model in r and below is the output of confusionMatrix I got- ...
0
votes
0answers
32 views

Multiclass classifier with undefined prediction, how to calculate metrices

I build a multiclass classifier. I want the classifier to predict a few samples with little false positives, rather then many samples with lots of false positives. Therefore I want to choose a ...
0
votes
0answers
31 views

How to choose metrics for evaluating classification results?

Recently we have developed a python library named PyCM specialized for analyzing multi-class confusion matrices. A parameter recommender system has been added in version 1.9 of this module in order ...
2
votes
1answer
227 views

How to interpret/deal with low precision confusion matrix?

I am developing a fraud detection model with unbalanced data set , I used Random Forests Algorithm. The confusion matrix showed that model has good sensitivity(recall) but low precision(Pos Pred ...
3
votes
0answers
27 views

Why is information gain rarely used as a reported effect size?

Commonly, in reporting effect sizes in medical articles, papers typically report: risk difference (RD), relative risk (RR) and odds ratio (OR) (and for variable timescale cases and other situations, ...
1
vote
1answer
58 views

Display Confusion Matrix properly -

I am trying to print the confusion matrix, but it is getting wrapped after few columns (or characters). I have tried several settings but didn't help: ...
0
votes
0answers
31 views

how to consider some miss classifications “half correct” in categorical_crossentropy - for a trading system

I have a trading system where the model receives 9 time-series and predict : A - strong down B - week down C - neutral D - week up E - strong up (these classes ...
1
vote
1answer
33 views

Shuffle vs Non-Shuffle - Confusion matrix reacts differently [closed]

Here is the config of my model : ...
1
vote
1answer
106 views

Can I compute an F1 score when the test data has no examples of one class?

I am working on a 3-class classification problem. We are cross-validating via a Leave-One-Out Approach, and there are some instances where the test data has no instances of one of my three classes. ...
0
votes
2answers
945 views

How to find cutoff point in Logistics Regression using R

I have run a Logistics Regression model in my data set. Below is the code: ...
0
votes
0answers
29 views

Optimal threshold for rejecting in classification model

Let's say I have a model to detect fake product. The model predict the probability whether the product is fake, with 0.0 being authentic, 1.0 is fake. Each true prediction (that product is not fake), ...
2
votes
1answer
431 views

Calculation of accuracy (and Cohen's kappa) using sensitivity, specificity, positive and negative predictive values

I read How to calculate specificity from accuracy and sensitivity, but I have two diagnostic performance measures more. Please correct me if I am wrong: if Sensitivity=TP/(TP+FN) Specificity=TN/(TN+...
1
vote
1answer
78 views

Estimating truth and confusion matrix from noisy observations with Expectation Maximization?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: $$...
1
vote
0answers
239 views

Using An ROC Curve to Evaluate a model

I have a number of questions on the ROC curves when being used to evaluate a model. My understanding of them is they can be used to determine the probability cutoff when classifying a row in a dataset ...
0
votes
1answer
16 views

Trying to read confusion matrices

I am given these 2 confusion matrix for a homework assignment but I am unsure how to interpret the data. I am asked to make a decision with these two matrices. For the first matrix am I interpreting ...
2
votes
0answers
81 views

Can you compute errors bars from Precision and Recall?

I am performing an object detection task for counting cars in an image. I have the confusion matrix (TP, FP, FP, TN) of the model. I guess TN is just zero in this case, as we aren't detecting where a ...
0
votes
1answer
356 views

If the position of 0 1 in confusion matrix changes does the formula for sensitivity changes

It may sounds very silly question, but I want to clear myself In confusion matrix we normally takes values like below: ...
1
vote
0answers
54 views

Rare event confusion matrix

Right now I'm gathering data collected from several individuals and manually classifying 100 randomly selected data points from each one. The condition I'm looking for is pretty rare. It occurs only ...
1
vote
1answer
355 views

How to determine whether a classifier is significantly better than random guessing?

I have a classifier Y that selects between three categories: A, B and C. I need to be able to quantitatively prove that my model is better (and by how much?), than a random classifier R that randomly ...
0
votes
1answer
2k views

Threshold to build confusion matrix?

I a have data set with 10 sections of data and each section shows one day observation. I designed the training and test set as follows: 8 sections for training the data and the last two sections for ...
4
votes
0answers
281 views

Can F1-Score be higher than accuracy?

I'm using sklearn's confusion_matrix and classification_report methods to compute the ...
6
votes
0answers
2k views

Confidence interval of precision / recall and F1 score

To summarise the predictive power of a classifier for end users, I'm using some metrics. However, as the users input data themselves, the amount of data and class distribution varies a lot. So to ...
1
vote
1answer
215 views

How to interpret that my model gives no negative class prediction on test set?

I am making multivariate time series classification with TUH seizure corpus dataset I have built this model with Keras, using LSTM layers : ...
4
votes
1answer
3k views

Is there any difference between Sensitivity and Recall?

In most of the places, I have found that sensitivity=recall. In terms of the Confusion Matrix, the formula for both of these is the same: $TP/(TP+FN)$. Is there any difference between these two ...
1
vote
1answer
83 views

Confidence interval for a macro/micro average

I'm working on a supervised multi class classifier that labels texts according to three possible classes. I calculated the one-vs-all precision, recall and F1 score for each class and the macro ...
-1
votes
1answer
221 views

Is it possible to get high sensitivity but low precision

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. This dataset is imbalanced with a majority examples belonging to normal class (0) and minority being class labeled ...
0
votes
1answer
309 views

Educational purpose :Calculation of sensitivity and specificity from confusion matrix for this example

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. The data labelled as 0 denote normal operating conditions and data labelled as ...
2
votes
2answers
2k views

Lack of understanding of LOOCV

I am trying to utilize LOOCV in the data partition in R. The idea of LOOCV is to train the model on n-1 set and test the model on the only remaining one set. Then, is to repeat this process n times ...
0
votes
1answer
634 views

multi label misclassification rates

I'm looking for a way to efficiently describe the performance of a multi-label classification model (if possible, something like confusion matrix for the multi-class classification). I'm not sure if ...
1
vote
2answers
455 views

analysing cross-entropy vs epochs curve

I am training a binary classification neural network model using matlab the graph that I got using 20 neurons in hidden layer is given below. the confusion matrix and graph between cross entropy vs ...