Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [precision-recall]

P&R are a way to measure the relevance of set of retrieved instances. Precision is the % of correct instances out of all instances retrieved. Relevance is the % of true instances retrieved. The harmonic mean of P&R is the F1-score. P&R are used in data mining to evaluate classifiers.

2
votes
1answer
27 views

Precision-Recall curve interpretation

When given an example confusion matrix: TP = 5000 FP = 1000 FN = 0 ...
0
votes
0answers
19 views

Best way to optimize recall?

So I built a model for a binary classification problem.My main class of interest if 'class 1'. False negatives are more costly for me, so it would make more sense to reduce false negatives by ...
2
votes
1answer
32 views

Balanced datasets are almost all predicted negative

Problem I am trying to do sentiment analysis using pretrained word vectors GloVe, which is essentially a look-up table that maps word to a fix-dimension vector. Since GloVe is initially designed to ...
0
votes
1answer
12 views

Does the threshold value of a logistic regression hypothesis has an effect on the accuracy?

It is true that the threshold value of a logistic regression hypothesis has an effect on the Precision/Recall metrics. Suppose you have trained a logistic regression classifier which is outputting $...
0
votes
1answer
16 views

How to interpret precision and recall for multiclass prediction?

I have a few models doing prediction with 4 classes, with the output precision and recall varying with different labels. For example I have (with the class labels being 0, 1, 2, 3 on the x axis): I ...
0
votes
1answer
29 views

Practical interpretation of Precision-Recall AUC

I have a classifier with an AUC (PR) of 0.06 which I will use for a practical interpretation. My test set consists of three months of data with a total of 2,200,000 observations of which 0.03 are ...
0
votes
0answers
14 views

Improveness given a certain AUPRC

I am training a machine learning model (Random Forest) for a multiclass problem (64 classes) in which most of them are highly imbalanced. That's why I am using mainly F1 score for checking the model's ...
0
votes
0answers
30 views

Precision and recall for movielens top n recommendation?

I know the definition. But because the real observed data is very sparse. I often come across the situation where recommended items are disjoint with the test data therefore I have 0 precision and ...
0
votes
1answer
20 views

What is the proper unit for F1? Is it a percentage?

F1-score is defined as $$ F_1 = \frac{2PR}{P+R}, $$ where P is precision [0..1] and R is recall [0..1]. My question is simply, is it right to describe F1 as a percentage? As in "our final F1-score ...
1
vote
1answer
46 views

Is it possible to have recall and precision of 0 while having an area under PR ~0.5?

As the title suggests, I am running a Random Forest classifier using Scala. To evaluate this classifier (and since I am handling highly imbalanced classes), I used the ...
1
vote
1answer
39 views

Optimising recall for multi-label classification?

I'm working on a multi-class multi-label classification problem where text (let's say comments on a website) should be assigned (possibly multiple) labels. There is a neutral (negative) class and ...
0
votes
1answer
40 views

Signs of Overfitting in Precision/Recall Curve

plz look at the following figures. As you cann see the precision is always 100% no matter which threshold (x-axis in logarithmic scale) you set! Also the second figure shows that we have a perfect ...
1
vote
2answers
85 views

AUPRC vs AUROC and updating training set in quasi-classification problem

I have an unbalanced classification problem (95% "0", 5% "1") regarding quality control."0" means "no problem" and "1" means "problem". I'm not predicting real cases one by one, this is, my client ...
0
votes
0answers
11 views

Matthews Correlation Component, but prioritizing recall over precision?

I am testing a binary classifier on a highly negative dataset and recently learned of the Matthews Correlation Coefficient (MCC). I had been using positive predictive value (precision) and true ...
0
votes
0answers
14 views

How to best evaluate the quality / precision of geo-spatial prediction?

I have developed an algorithm to predict locations of certain items in 2D-space. The input data consists of various fuzzy / blurred observations of those items. Therefore, I am not performing a binary ...
0
votes
0answers
28 views

What is the difference between Recall and Coverage?

I am unfamiliar with Coverage - and in the past thought it were simply a less precise referral to Recall. However I do see ...
0
votes
0answers
17 views

How to obtain all the true positives given two sets of precision and recall values?

Given that I have a dataset with some number of 1s (positives) and 0s (negatives), and the following two sets of precision and recall values: Set A: Positive precision: 0.51 Positive recall: 0.8 ...
1
vote
1answer
46 views

Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is ...
0
votes
1answer
20 views

How to calculate ± deviation for precision and recall?

I found a precision and recall report table like as below Precision Recall .470±.009 .934±.013 .239±.010 .610±.013 I need the guidelines for ±.009 and ±.013 ...
0
votes
0answers
15 views

Highly unbalanced data set .Minority Class 1 %

I want to optimize precision and recall i.e f-score but I want to keep high precision . What are the possible ways of doing binary classification on such imbalanced data set [Minority class 1 %]. I'...
0
votes
1answer
45 views

High Precision and low recall score for TF-IDF when using KNN algorithm

I have twitter data which is labelled with the sentiment(Postive, Negative, Neutral) and I have evaluated the performance of Tf-Idf and Doc2Vec feature extractor using the KNN algorithm and logistic ...
0
votes
0answers
11 views

How to calculate precision-recall curve for multiple binary classifier for multiple classes

I am using autoencoder based multiple binary classifier for text regeneration each trained on data related to multiple classes. In other words, each classifier is trained to classify only one class. I ...
0
votes
0answers
12 views

Calculating the number of true positives from a precision-recall curve

Using the below precision recall graph where recall is on x-axis and precision is on y-axis can I use this formula to calculate the number of predictions for a given precision, recall threshold ? ...
0
votes
1answer
28 views

Adjusting precision recall curve for oversampling

I built a model for a binary target using oversampled data. The population target prevalence is 0.25. I oversampled to 0.5 by keeping the entirety of the minority class and sampling a portion of the ...
0
votes
0answers
4 views

Precision metric over a subset of classes?

I have a classification problem where I want to measure the precision over only a subset of classes. That is, for each time the model predicts a certain class within the subset, what is it’s precision?...
0
votes
0answers
16 views

Calculating F1 with crossvalidation - Average over folds or compute at end?

When performing crossvalidation, is it appropriate to calculate metrics (e.g. accuracy, F1) for each fold and then average these, or to calculate them at the end using the prediction results from the ...
0
votes
0answers
26 views

How much “ground truth” data do I need to evaluate a heuristic classification approach?

I have a heuristic approach to a de-duplication problem: There are about 300 million unknown data points. I attempt to match each data point with one of about 4 million independently known data ...
0
votes
0answers
11 views

What values should the Interpolated 11-point precision curve, measured at k, have when P@k = 0?

I'm evaluating a few retrieval models using the 11-point Interpolated precision x recall curve (here is a description of the protocol I've been using: https://nlp.stanford.edu/IR-book/html/htmledition/...
0
votes
0answers
10 views

Which performance metric to use for stratified data? [duplicate]

I'm trying to classify a data into 3 classes (supervised), one of which is heavily underrepresented in the data set. In order to combat this imbalance, I decided to stratify the data. Now I want to ...
1
vote
1answer
26 views

Is recall a relatively meaningless metric in a balanced dataset?

Just looking for a sanity check here. Leaving aside precision, is talking about the recall of a binary classification algorithm sensible where 50% of the cases presented to it are positive and 50% ...
0
votes
0answers
11 views

When do we use precision/recall over accuracy when evaluating a classifier? [duplicate]

When do we use precision/recall over accuracy when evaluating a classifier? What kind of scenarios would mean precision/recall metrics would be better than accuracy?
0
votes
1answer
349 views

Am I allowed to average the list of precision and recall after k-fold cross validation?

I have created a 5-fold cross validation model and used cross_val_score function to calculate the precision and recall of the cross validated model as follows: ...
2
votes
1answer
34 views

Which metric to use in an ordering problem? auPR / ROC / Lift?

I need to order Users from most likely to perform a binary action X in the next n days, to ...
0
votes
1answer
64 views

ROC and PR curves after over/under sampling in Unbalanced datasets

As I understood till now, ROC curves are not a good presentation of unbalanced datasets and PR curves are preferred because ROC curves are not sensitive to false positives. If we now use resample ...
0
votes
1answer
112 views

High precision or High recall [duplicate]

I have a question about precision and recall. Which classifier is better, one with high precision or high recall for medical purposes like finding patients with allergy? Can we use F-measure for ...
0
votes
0answers
19 views

Evaluating binary classifier model. What can say precision, recall etc.? [duplicate]

i'm trying to understand wether my model has good performance or not. I have binary classifier for summarization sentences: important or not (extractive approach) on specific corpus. Dataset is ...
1
vote
1answer
183 views

calculating overall error in k-fold cross validation

when using k-fold cross validation i thought the overall error was equal to the mean of errors of each fold. the error being anything from MAE and RMSE to NDCG,F-measure, precision and recall. however ...
2
votes
2answers
49 views

Does precision or recall have more importance? Are they to be considered equivalent measures of accuracy?

Does precision or recall have more importance? Or are they to be considered equivalent measures of accuracy? They can produce different numbers; they refer to slightly different errors. But is it ...
2
votes
0answers
46 views

My roc is low while precision and recall are high.Why is it so?

I bulit a naive bayes classifier from 60k vectors of text and each of the text is a 2000 dimension vector(Used bag of words for vectorization). Used simple cross validator to find the best alpha and ...
0
votes
1answer
158 views

The representation of F1-score on the Precision-Recall Curve

Is there a way to project the F1-score on the precision-recall curve for a such binary classifier? Is there a relationship between the area under the precision-recall curve and F1-score? ...
0
votes
0answers
23 views

Understanding model performance

I built a multiclass (and very imbalanced classes) classifier. When evaluating I found an average F1 of .98 and the classifier seems to be working rather well. However, on evaluating the ROC and ...
0
votes
0answers
91 views

Best performance metric for highly imbalanced dataset f1 score vs kappa vs AUROC

I have highly imbalanced data (like fraud detection). I usually use f1 score to evaluate model performance. But I also saw people recommend AUROC and cohen's kappa. I'm seeking expert opinion on what ...
0
votes
2answers
35 views

Plot ROC or PR curves from either the X,Y coordinates (i.e TPR/TNR; or PPV/TPR) or list of predictions (class probabilities)? [closed]

I have a list of X,Y coordinates for plotting both a ROC curve and a PR curve. I also have the data which was used to calculate those coordinates (i.e. a list of individual predictions with binary ...
0
votes
0answers
34 views

which recall value to plot for same precision in PR curve?

Suppose, after sorting the true labels by the corresponding classifier scores, we obtain the following: $$[False, True, False, True, True, True, False, False],$$ which leads to the following points ...
1
vote
0answers
32 views

Computing recall on classification task

I have done google searches on computing "recall @ X" several times in the last few months, and every page I read gives me the same answer: $\frac{TP}{TP + FN}$ ... but the story I get from my PI and ...
0
votes
0answers
41 views

What a convex Precision-Recall curve means for training dataset?

Situation I have trained a GBDT model(gradient-boosted decision tree, a tree ensemble model) with a training dataset, and when I calculate PR curve on the same training set, it looks convex: For ...
0
votes
0answers
14 views

what happened if we reverse a fraud classification model?

Let us assume we train a classification model on fraud data sets to detect fraud and no fraud. The classification model threshold has high precision and low recall with ROC AUC=0.9, so we will only ...
0
votes
1answer
22 views

Precision and Recall: Valid Combination?

I'm looking at a precision-recall curve for a binary classification task. My precision-recall curve intersects the y-axis (precision) at 60% and the x-axis at 15%. So I get 15% precision at 100% ...
4
votes
0answers
970 views

Confidence interval of precision / recall and F1 score

To summarise the predictive power of a classifier for end users, I'm using some metrics. However, as the users input data themselves, the amount of data and class distribution varies a lot. So to ...
3
votes
1answer
1k views

Is there any difference between Sensitivity and Recall?

In most of the places, I have found that sensitivity=recall. In terms of the Confusion Matrix, the formula for both of these is the same: $TP/(TP+FN)$. Is there any difference between these two ...