Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [confusion-matrix]

A confusion matrix is a special contingency table used to evaluate the predictive accuracy of a classifier. Predicted classes are listed in rows & actual classes in columns, w/ counts of respective cases in each cell.

0
votes
0answers
7 views

Compute standard deviation of accuracy

The following code represents my problem : ...
0
votes
0answers
14 views

Confusion matrix and accuracy glmnet [on hold]

I have this code for a LASSO regression in R: ...
1
vote
1answer
7 views

Is it appropriate to use a confusion matrix for frequency data?

I am testing the accuracy of a machine learning approach that counts cars in images. I have both a predicted dataset and a "real" dataset that was generated by a human. For example, this is what my ...
0
votes
1answer
23 views

Signs of Overfitting in Precision/Recall Curve

plz look at the following figures. As you cann see the precision is always 100% no matter which threshold (x-axis in logarithmic scale) you set! Also the second figure shows that we have a perfect ...
1
vote
1answer
31 views

R - confusionMatrix function in R [duplicate]

I am working on a model in r and below is the output of confusionMatrix I got- ...
0
votes
0answers
12 views

Multiclass classifier with undefined prediction, how to calculate metrices

I build a multiclass classifier. I want the classifier to predict a few samples with little false positives, rather then many samples with lots of false positives. Therefore I want to choose a ...
0
votes
0answers
27 views

How to choose metrics for evaluating classification results?

Recently we have developed a python library named PyCM specialized for analyzing multi-class confusion matrices. A parameter recommender system has been added in version 1.9 of this module in order ...
2
votes
1answer
54 views

How to interpret/deal with low precision confusion matrix?

I am developing a fraud detection model with unbalanced data set , I used Random Forests Algorithm. The confusion matrix showed that model has good sensitivity(recall) but low precision(Pos Pred ...
3
votes
0answers
21 views

Why is information gain rarely used as a reported effect size?

Commonly, in reporting effect sizes in medical articles, papers typically report: risk difference (RD), relative risk (RR) and odds ratio (OR) (and for variable timescale cases and other situations, ...
1
vote
1answer
22 views

Display Confusion Matrix properly -

I am trying to print the confusion matrix, but it is getting wrapped after few columns (or characters). I have tried several settings but didn't help: ...
0
votes
0answers
25 views

how to consider some miss classifications “half correct” in categorical_crossentropy - for a trading system

I have a trading system where the model receives 9 time-series and predict : A - strong down B - week down C - neutral D - week up E - strong up (these classes ...
1
vote
1answer
26 views
0
votes
1answer
26 views

Can I compute an F1 score when the test data has no examples of one class?

I am working on a 3-class classification problem. We are cross-validating via a Leave-One-Out Approach, and there are some instances where the test data has no instances of one of my three classes. ...
0
votes
2answers
162 views

How to find cutoff point in Logistics Regression using R

I have run a Logistics Regression model in my data set. Below is the code: ...
0
votes
0answers
23 views

Optimal threshold for rejecting in classification model

Let's say I have a model to detect fake product. The model predict the probability whether the product is fake, with 0.0 being authentic, 1.0 is fake. Each true prediction (that product is not fake), ...
0
votes
0answers
43 views

Confusion regarding confusion matrix

Suppose we are trying to predict the outcome of a test which can be 0 or 1. The below is the confusion matrix: Now, if we are interested in outcome=1, the Sensitivity = $\frac{10}{10+20} = \frac{10}{...
1
vote
0answers
211 views

Calculation of accuracy (and Cohen's kappa) using sensitivity, specificity, positive and negative predictive values

I read How to calculate specificity from accuracy and sensitivity, but I have two diagnostic performance measures more. Please correct me if I am wrong: if Sensitivity=TP/(TP+FN) Specificity=TN/(TN+...
1
vote
1answer
53 views

Estimating truth and confusion matrix from noisy observations with Expectation Maximization?

Suppose we have $m$ sources, each of which noisily observe the same set of $n$ independent events from the outcome set $\{A,B,C\}$. Each source has a confusion matrix, for example for source $i$: $$...
1
vote
0answers
121 views

Using An ROC Curve to Evaluate a model

I have a number of questions on the ROC curves when being used to evaluate a model. My understanding of them is they can be used to determine the probability cutoff when classifying a row in a dataset ...
0
votes
1answer
14 views

Trying to read confusion matrices

I am given these 2 confusion matrix for a homework assignment but I am unsure how to interpret the data. I am asked to make a decision with these two matrices. For the first matrix am I interpreting ...
2
votes
0answers
49 views

Can you compute errors bars from Precision and Recall?

I am performing an object detection task for counting cars in an image. I have the confusion matrix (TP, FP, FP, TN) of the model. I guess TN is just zero in this case, as we aren't detecting where a ...
0
votes
1answer
121 views

If the position of 0 1 in confusion matrix changes does the formula for sensitivity changes

It may sounds very silly question, but I want to clear myself In confusion matrix we normally takes values like below: ...
1
vote
0answers
22 views

Rare event confusion matrix

Right now I'm gathering data collected from several individuals and manually classifying 100 randomly selected data points from each one. The condition I'm looking for is pretty rare. It occurs only ...
0
votes
0answers
14 views

Using confusion matrix to adjust prediction counts

We have a CNN that classifies images into around 21 classes. In our study, we are interested in the proportion of images in each class from a particular sample of say 2000 images. Of course the ...
0
votes
0answers
52 views

Interpretation of F1 or DOR for confusion matrix

I am using some data to validate a binary classfier. I have produced the first 103 manual classifications and I'm checking to see how I should interpret my results. First I populated my confusion ...
1
vote
1answer
187 views

How to determine whether a classifier is significantly better than random guessing?

I have a classifier Y that selects between three categories: A, B and C. I need to be able to quantitatively prove that my model is better (and by how much?), than a random classifier R that randomly ...
0
votes
1answer
712 views

Threshold to build confusion matrix?

I a have data set with 10 sections of data and each section shows one day observation. I designed the training and test set as follows: 8 sections for training the data and the last two sections for ...
4
votes
0answers
136 views

Can F1-Score be higher than accuracy?

I'm using sklearn's confusion_matrix and classification_report methods to compute the ...
3
votes
0answers
695 views

Confidence interval of precision / recall and F1 score

To summarise the predictive power of a classifier for end users, I'm using some metrics. However, as the users input data themselves, the amount of data and class distribution varies a lot. So to ...
1
vote
1answer
190 views

How to interpret that my model gives no negative class prediction on test set?

I am making multivariate time series classification with TUH seizure corpus dataset I have built this model with Keras, using LSTM layers : ...
0
votes
1answer
872 views

Is there any difference between Sensitivity and Recall?

In most of the places, I have found that sensitivity=recall. In terms of the Confusion Matrix, the formula for both of these is the same: $TP/(TP+FN)$. Is there any difference between these two ...
0
votes
0answers
15 views

How do I know the margin of error of my predicted values?

Questions will be written using bold formatting: I have come up with a model on R that gave me the following output: I've been asked to create a confusion matrix in order to compare my predicted ...
0
votes
0answers
48 views

Confidence interval for a macro/micro average

I'm working on a supervised multi class classifier that labels texts according to three possible classes. I calculated the one-vs-all precision, recall and F1 score for each class and the macro ...
-1
votes
1answer
101 views

Is it possible to get high sensitivity but low precision

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. This dataset is imbalanced with a majority examples belonging to normal class (0) and minority being class labeled ...
0
votes
1answer
151 views

Educational purpose :Calculation of sensitivity and specificity from confusion matrix for this example

Consider a case where the number of labelled data as 0 = 1400 and labelled as 1 =100. The data labelled as 0 denote normal operating conditions and data labelled as ...
0
votes
0answers
33 views

Can we use precision and recall to evaluate text prediction model results?

I am using an LSTM model to predict/complete words from a seed input. These words or tokens represent xml markups or fields. So, as an example, I give my model a seed input : ...
2
votes
2answers
968 views

Lack of understanding of LOOCV

I am trying to utilize LOOCV in the data partition in R. The idea of LOOCV is to train the model on n-1 set and test the model on the only remaining one set. Then, is to repeat this process n times ...
0
votes
1answer
475 views

multi label misclassification rates

I'm looking for a way to efficiently describe the performance of a multi-label classification model (if possible, something like confusion matrix for the multi-class classification). I'm not sure if ...
1
vote
2answers
261 views

analysing cross-entropy vs epochs curve

I am training a binary classification neural network model using matlab the graph that I got using 20 neurons in hidden layer is given below. the confusion matrix and graph between cross entropy vs ...
0
votes
1answer
534 views

How to correctly read a classification report?

Firstly, is there a difference between model performance and it's accuracy? If yes, what exactly? Secondly, what can I interpret from this classification_report of ...
0
votes
0answers
42 views

Corrupted inputs, better accuracy?

I am training a neural network for audio classification using the UrbanSound8K dataset. I want to study how different intrusions in the inputs affect the network predictability. One such intrusion ...
1
vote
2answers
258 views

Accuracy computation with clustering in 3x3 matrix

I have three true class, A, B, C in my dataset. and I got 3 clusters (0, 1, 2) from the clustering algorithm. They are not supposed to belong to the same class. For E.g. cluster 1 can belong to class ...
1
vote
0answers
98 views

Statistical test for confounding bias in confusion matrix

I have a confusion matrix with on the X-axis the predicted class and on the Y-axis the true class. Now there seems to be some bias between some classes, especially between class number 7 and 11. Now I ...
1
vote
1answer
434 views

How to understand 3X3 confusion matrix in R from prediction results?

I am trying to cluster my data points in 3 groups using k-means for a time series. Let's say at time t=T, I have 3 clusters A, B, and C. Taking this as reference, I march forward in time cluster the ...
4
votes
2answers
295 views

Calculate classifier accuracy from per label accuracy

I would like to have a per Label accuracy and classifier level accuracy, but my calculations seem incorrect. Here is my full example. Let's say I have a multilabel classifier which predicted in the ...
0
votes
0answers
54 views

How to compute accuracy score for principal component regression in R?

I want to know whether we can calculate accuracy for principal component regression in R. The Target variable has only two values 0 or 1. I tried factoring training $ Target and validation $ Target ...
1
vote
0answers
24 views

classifier metric that takes into account the nature of errors?

I have an SVM classifier that gives a confusion matrix like below: I am looking for some way to quantify its performance which takes into account the fact that many of the errors are close to being ...
19
votes
2answers
5k views

FPR (false positive rate) vs FDR (false discovery rate)

The following quote comes from the famous research paper Statistical significance for genome wide studies by Storey & Tibshirani (2003): For example, a false positive rate of 5% means that on ...
0
votes
0answers
191 views

Confusion matrix interpretation

I try to classify images (characters A-Z) with Gaussian Mixture Model with PCA reduction 95%. To do so, I instantiate a GMM per label, train the GMMs (86000 samples) and then test them (20000 samples)....