Questions tagged [confusion-matrix]

A confusion matrix is a contingency table used to evaluate the predictive accuracy of a classifier. Confusion matrix is the 2x2 frequency table with counts "True positive", "True negative", "False positive", "False negative", relating classifying to a class of interest vs. else class. But in a broader sense, any frequency kxk crosstabulation "Predicted" x "Actual" classes can be called a confusion matrix, in the context of evaluation of a classifier.

Filter by
Sorted by
Tagged with
0
votes
0answers
29 views

Precision, Recall, F1 Score and multiclass confusion matrix

My goal is to evaluate some clustering algorithms. Lets say that I’ve got one set o elements: { 1 ; 2; 3; 4; 5; 6; 7; 8 }. This set was grouped by human into two groups: (1 ; 2; 3; 4; 5; 6; 8) and (7)....
0
votes
0answers
24 views

Understanding ratio between precision and recall

I have a naive question regarding the ratio of precision and recall. When I build the model, I am able to get precision and recall. Later, I could use this model to make perditions, which is ...
2
votes
1answer
50 views
2
votes
1answer
32 views

Precision-Recall curve interpretation

When given an example confusion matrix: TP = 5000 FP = 1000 FN = 0 ...
3
votes
1answer
86 views

Find true negatives in a confusion matrix

I'm trying to find the True negative in a confusion matrix, I have computed successfully from scratch the precision and recall/sensibility, now i need to compute the accuracy and specificity. This is ...
0
votes
0answers
43 views

Is there a name for “normalized accuracy” as a statistic?

In short: I'm using a statistic representing the "normalized accuracy" of a confusion matrix. Is there a formal term for this? $$\text{normacc} = \frac{\text{acc}-\text{thacc}}{1-\text{thacc}}$$ ...
0
votes
0answers
17 views

How to understand 2X2 confusion matrix one-r?

with this data set when applying one-r with weka choose age group: but I do not understand how weka made this confusion matrix
0
votes
1answer
34 views

Is it possible to estimate accuracy, precision and recall with the given data?

Background: I talked to my friend today and according to herm(him/her) I can calculate precision, recall and accuracy with the current information. Total instances T: 19,532. Instances belonging to ...
0
votes
1answer
121 views

Compute standard deviation of accuracy

edit - more information about what the code given should represent The following pseudocode outlines the problem as I have it ...
1
vote
1answer
13 views

Is it appropriate to use a confusion matrix for frequency data?

I am testing the accuracy of a machine learning approach that counts cars in images. I have both a predicted dataset and a "real" dataset that was generated by a human. For example, this is what my ...
0
votes
1answer
61 views

Signs of Overfitting in Precision/Recall Curve

plz look at the following figures. As you cann see the precision is always 100% no matter which threshold (x-axis in logarithmic scale) you set! Also the second figure shows that we have a perfect ...
1
vote
1answer
55 views
1
vote
1answer
273 views

R - confusionMatrix function in R [duplicate]

I am working on a model in r and below is the output of confusionMatrix I got- ...
0
votes
0answers
22 views

Multiclass classifier with undefined prediction, how to calculate metrices

I build a multiclass classifier. I want the classifier to predict a few samples with little false positives, rather then many samples with lots of false positives. Therefore I want to choose a ...
0
votes
0answers
30 views

How to choose metrics for evaluating classification results?

Recently we have developed a python library named PyCM specialized for analyzing multi-class confusion matrices. A parameter recommender system has been added in version 1.9 of this module in order ...
2
votes
1answer
131 views

How to interpret/deal with low precision confusion matrix?

I am developing a fraud detection model with unbalanced data set , I used Random Forests Algorithm. The confusion matrix showed that model has good sensitivity(recall) but low precision(Pos Pred ...
3
votes
0answers
25 views

Why is information gain rarely used as a reported effect size?

Commonly, in reporting effect sizes in medical articles, papers typically report: risk difference (RD), relative risk (RR) and odds ratio (OR) (and for variable timescale cases and other situations, ...
1
vote
1answer
30 views

Display Confusion Matrix properly -

I am trying to print the confusion matrix, but it is getting wrapped after few columns (or characters). I have tried several settings but didn't help: ...
0
votes
0answers
29 views

how to consider some miss classifications “half correct” in categorical_crossentropy - for a trading system

I have a trading system where the model receives 9 time-series and predict : A - strong down B - week down C - neutral D - week up E - strong up (these classes ...
1
vote
1answer
30 views

Shuffle vs Non-Shuffle - Confusion matrix reacts differently [closed]

Here is the config of my model : ...
1
vote
1answer
67 views

Can I compute an F1 score when the test data has no examples of one class?

I am working on a 3-class classification problem. We are cross-validating via a Leave-One-Out Approach, and there are some instances where the test data has no instances of one of my three classes. ...
0
votes
2answers
546 views

How to find cutoff point in Logistics Regression using R

I have run a Logistics Regression model in my data set. Below is the code: ...
0
votes
0answers
29 views

Optimal threshold for rejecting in classification model

Let's say I have a model to detect fake product. The model predict the probability whether the product is fake, with 0.0 being authentic, 1.0 is fake. Each true prediction (that product is not fake), ...
0
votes
0answers
50 views

Confusion regarding confusion matrix

Suppose we are trying to predict the outcome of a test which can be 0 or 1. The below is the confusion matrix: Now, if we are interested in outcome=1, the Sensitivity = $\frac{10}{10+20} = \frac{10}{...
2
votes
0answers
338 views

Calculation of accuracy (and Cohen's kappa) using sensitivity, specificity, positive and negative predictive values

I read How to calculate specificity from accuracy and sensitivity, but I have two diagnostic performance measures more. Please correct me if I am wrong: if Sensitivity=TP/(TP+FN) Specificity=TN/(TN+...
1
vote
1answer
65 views

Estimating truth and confusion matrix from noisy observations with Expectation Maximization?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: $$...
1
vote
0answers
205 views

Using An ROC Curve to Evaluate a model

I have a number of questions on the ROC curves when being used to evaluate a model. My understanding of them is they can be used to determine the probability cutoff when classifying a row in a dataset ...
0
votes
1answer
16 views

Trying to read confusion matrices

I am given these 2 confusion matrix for a homework assignment but I am unsure how to interpret the data. I am asked to make a decision with these two matrices. For the first matrix am I interpreting ...
2
votes
0answers
69 views

Can you compute errors bars from Precision and Recall?

I am performing an object detection task for counting cars in an image. I have the confusion matrix (TP, FP, FP, TN) of the model. I guess TN is just zero in this case, as we aren't detecting where a ...
0
votes
1answer
237 views

If the position of 0 1 in confusion matrix changes does the formula for sensitivity changes

It may sounds very silly question, but I want to clear myself In confusion matrix we normally takes values like below: ...
1
vote
0answers
37 views

Rare event confusion matrix

Right now I'm gathering data collected from several individuals and manually classifying 100 randomly selected data points from each one. The condition I'm looking for is pretty rare. It occurs only ...
0
votes
0answers
19 views

Using confusion matrix to adjust prediction counts

We have a CNN that classifies images into around 21 classes. In our study, we are interested in the proportion of images in each class from a particular sample of say 2000 images. Of course the ...
1
vote
1answer
258 views

How to determine whether a classifier is significantly better than random guessing?

I have a classifier Y that selects between three categories: A, B and C. I need to be able to quantitatively prove that my model is better (and by how much?), than a random classifier R that randomly ...
0
votes
1answer
1k views

Threshold to build confusion matrix?

I a have data set with 10 sections of data and each section shows one day observation. I designed the training and test set as follows: 8 sections for training the data and the last two sections for ...
4
votes
0answers
210 views

Can F1-Score be higher than accuracy?

I'm using sklearn's confusion_matrix and classification_report methods to compute the ...
5
votes
0answers
1k views

Confidence interval of precision / recall and F1 score

To summarise the predictive power of a classifier for end users, I'm using some metrics. However, as the users input data themselves, the amount of data and class distribution varies a lot. So to ...
1
vote
1answer
202 views

How to interpret that my model gives no negative class prediction on test set?

I am making multivariate time series classification with TUH seizure corpus dataset I have built this model with Keras, using LSTM layers : ...
3
votes
1answer
2k views

Is there any difference between Sensitivity and Recall?

In most of the places, I have found that sensitivity=recall. In terms of the Confusion Matrix, the formula for both of these is the same: $TP/(TP+FN)$. Is there any difference between these two ...
1
vote
1answer
62 views

Confidence interval for a macro/micro average

I'm working on a supervised multi class classifier that labels texts according to three possible classes. I calculated the one-vs-all precision, recall and F1 score for each class and the macro ...
-1
votes
1answer
152 views

Is it possible to get high sensitivity but low precision

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. This dataset is imbalanced with a majority examples belonging to normal class (0) and minority being class labeled ...
0
votes
1answer
233 views

Educational purpose :Calculation of sensitivity and specificity from confusion matrix for this example

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. The data labelled as 0 denote normal operating conditions and data labelled as ...
2
votes
2answers
1k views

Lack of understanding of LOOCV

I am trying to utilize LOOCV in the data partition in R. The idea of LOOCV is to train the model on n-1 set and test the model on the only remaining one set. Then, is to repeat this process n times ...
0
votes
1answer
579 views

multi label misclassification rates

I'm looking for a way to efficiently describe the performance of a multi-label classification model (if possible, something like confusion matrix for the multi-class classification). I'm not sure if ...
1
vote
2answers
333 views

analysing cross-entropy vs epochs curve

I am training a binary classification neural network model using matlab the graph that I got using 20 neurons in hidden layer is given below. the confusion matrix and graph between cross entropy vs ...
0
votes
1answer
821 views

How to correctly read a classification report?

Firstly, is there a difference between model performance and it's accuracy? If yes, what exactly? Secondly, what can I interpret from this classification_report of ...
0
votes
0answers
44 views

Corrupted inputs, better accuracy?

I am training a neural network for audio classification using the UrbanSound8K dataset. I want to study how different intrusions in the inputs affect the network predictability. One such intrusion ...
1
vote
2answers
400 views

Accuracy computation with clustering in 3x3 matrix

I have three true class, A, B, C in my dataset. and I got 3 clusters (0, 1, 2) from the clustering algorithm. They are not supposed to belong to the same class. For E.g. cluster 1 can belong to class ...
1
vote
0answers
145 views

Statistical test for confounding bias in confusion matrix

I have a confusion matrix with on the X-axis the predicted class and on the Y-axis the true class. Now there seems to be some bias between some classes, especially between class number 7 and 11. Now I ...
1
vote
1answer
470 views

How to understand 3X3 confusion matrix in R from prediction results?

I am trying to cluster my data points in 3 groups using k-means for a time series. Let's say at time t=T, I have 3 clusters A, B, and C. Taking this as reference, I march forward in time cluster the ...
4
votes
2answers
450 views

Calculate classifier accuracy from per label accuracy

I would like to have a per Label accuracy and classifier level accuracy, but my calculations seem incorrect. Here is my full example. Let's say I have a multilabel classifier which predicted in the ...