Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [precision-recall]

P&R are a way to measure the relevance of set of retrieved instances. Precision is the % of correct instances out of all instances retrieved. Relevance is the % of true instances retrieved. The harmonic mean of P&R is the F1-score. P&R are used in data mining to evaluate classifiers.

1
vote
1answer
32 views

Is it possible to have recall and precision of 0 while having an area under PR ~0.5?

As the title suggests, I am running a Random Forest classifier using Scala. To evaluate this classifier (and since I am handling highly imbalanced classes), I used the ...
1
vote
1answer
19 views

Optimising recall for multi-label classification?

I'm working on a multi-class multi-label classification problem where text (let's say comments on a website) should be assigned (possibly multiple) labels. There is a neutral (negative) class and ...
0
votes
1answer
23 views

Signs of Overfitting in Precision/Recall Curve

plz look at the following figures. As you cann see the precision is always 100% no matter which threshold (x-axis in logarithmic scale) you set! Also the second figure shows that we have a perfect ...
1
vote
2answers
52 views

AUPRC vs AUROC and updating training set in quasi-classification problem

I have an unbalanced classification problem (95% "0", 5% "1") regarding quality control."0" means "no problem" and "1" means "problem". I'm not predicting real cases one by one, this is, my client ...
0
votes
0answers
10 views

Matthews Correlation Component, but prioritizing recall over precision?

I am testing a binary classifier on a highly negative dataset and recently learned of the Matthews Correlation Coefficient (MCC). I had been using positive predictive value (precision) and true ...
0
votes
0answers
12 views

How to best evaluate the quality / precision of geo-spatial prediction?

I have developed an algorithm to predict locations of certain items in 2D-space. The input data consists of various fuzzy / blurred observations of those items. Therefore, I am not performing a binary ...
0
votes
0answers
11 views

What is the difference between Recall and Coverage?

I am unfamiliar with Coverage - and in the past thought it were simply a less precise referral to Recall. However I do see ...
0
votes
0answers
17 views

How to obtain all the true positives given two sets of precision and recall values?

Given that I have a dataset with some number of 1s (positives) and 0s (negatives), and the following two sets of precision and recall values: Set A: Positive precision: 0.51 Positive recall: 0.8 ...
1
vote
1answer
34 views

Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is ...
0
votes
1answer
18 views

How to calculate ± deviation for precision and recall?

I found a precision and recall report table like as below Precision Recall .470±.009 .934±.013 .239±.010 .610±.013 I need the guidelines for ±.009 and ±.013 ...
0
votes
0answers
11 views

Highly unbalanced data set .Minority Class 1 %

I want to optimize precision and recall i.e f-score but I want to keep high precision . What are the possible ways of doing binary classification on such imbalanced data set [Minority class 1 %]. I'...
0
votes
0answers
26 views

Model Performance using Precision as evaluation metric

I am dealing with an imbalanced class with the following distribution : (Total dataset size : 10763 X 20) 0 : 91% 1 : 9% To build model on this dataset having class imbalance, I have compared ...
0
votes
1answer
21 views

High Precision and low recall score for TF-IDF when using KNN algorithm

I have twitter data which is labelled with the sentiment(Postive, Negative, Neutral) and I have evaluated the performance of Tf-Idf and Doc2Vec feature extractor using the KNN algorithm and logistic ...
0
votes
0answers
8 views

How to calculate precision-recall curve for multiple binary classifier for multiple classes

I am using autoencoder based multiple binary classifier for text regeneration each trained on data related to multiple classes. In other words, each classifier is trained to classify only one class. I ...
0
votes
0answers
12 views

Calculating the number of true positives from a precision-recall curve

Using the below precision recall graph where recall is on x-axis and precision is on y-axis can I use this formula to calculate the number of predictions for a given precision, recall threshold ? ...
0
votes
1answer
17 views

Adjusting precision recall curve for oversampling

I built a model for a binary target using oversampled data. The population target prevalence is 0.25. I oversampled to 0.5 by keeping the entirety of the minority class and sampling a portion of the ...
0
votes
0answers
4 views

Precision metric over a subset of classes?

I have a classification problem where I want to measure the precision over only a subset of classes. That is, for each time the model predicts a certain class within the subset, what is it’s precision?...
0
votes
0answers
12 views

Calculating F1 with crossvalidation - Average over folds or compute at end?

When performing crossvalidation, is it appropriate to calculate metrics (e.g. accuracy, F1) for each fold and then average these, or to calculate them at the end using the prediction results from the ...
0
votes
0answers
24 views

How much “ground truth” data do I need to evaluate a heuristic classification approach?

I have a heuristic approach to a de-duplication problem: There are about 300 million unknown data points. I attempt to match each data point with one of about 4 million independently known data ...
0
votes
0answers
4 views

What values should the Interpolated 11-point precision curve, measured at k, have when P@k = 0?

I'm evaluating a few retrieval models using the 11-point Interpolated precision x recall curve (here is a description of the protocol I've been using: https://nlp.stanford.edu/IR-book/html/htmledition/...
0
votes
0answers
9 views

Which performance metric to use for stratified data? [duplicate]

I'm trying to classify a data into 3 classes (supervised), one of which is heavily underrepresented in the data set. In order to combat this imbalance, I decided to stratify the data. Now I want to ...
1
vote
1answer
26 views

Is recall a relatively meaningless metric in a balanced dataset?

Just looking for a sanity check here. Leaving aside precision, is talking about the recall of a binary classification algorithm sensible where 50% of the cases presented to it are positive and 50% ...
0
votes
0answers
11 views

When do we use precision/recall over accuracy when evaluating a classifier? [duplicate]

When do we use precision/recall over accuracy when evaluating a classifier? What kind of scenarios would mean precision/recall metrics would be better than accuracy?
0
votes
1answer
139 views

Am I allowed to average the list of precision and recall after k-fold cross validation?

I have created a 5-fold cross validation model and used cross_val_score function to calculate the precision and recall of the cross validated model as follows: ...
2
votes
1answer
29 views

Which metric to use in an ordering problem? auPR / ROC / Lift?

I need to order Users from most likely to perform a binary action X in the next n days, to ...
0
votes
1answer
48 views

ROC and PR curves after over/under sampling in Unbalanced datasets

As I understood till now, ROC curves are not a good presentation of unbalanced datasets and PR curves are preferred because ROC curves are not sensitive to false positives. If we now use resample ...
0
votes
1answer
34 views

High precision or High recall [duplicate]

I have a question about precision and recall. Which classifier is better, one with high precision or high recall for medical purposes like finding patients with allergy? Can we use F-measure for ...
0
votes
0answers
19 views

Evaluating binary classifier model. What can say precision, recall etc.? [duplicate]

i'm trying to understand wether my model has good performance or not. I have binary classifier for summarization sentences: important or not (extractive approach) on specific corpus. Dataset is ...
1
vote
1answer
139 views

calculating overall error in k-fold cross validation

when using k-fold cross validation i thought the overall error was equal to the mean of errors of each fold. the error being anything from MAE and RMSE to NDCG,F-measure, precision and recall. however ...
2
votes
2answers
44 views

Does precision or recall have more importance? Are they to be considered equivalent measures of accuracy?

Does precision or recall have more importance? Or are they to be considered equivalent measures of accuracy? They can produce different numbers; they refer to slightly different errors. But is it ...
2
votes
0answers
43 views

My roc is low while precision and recall are high.Why is it so?

I bulit a naive bayes classifier from 60k vectors of text and each of the text is a 2000 dimension vector(Used bag of words for vectorization). Used simple cross validator to find the best alpha and ...
0
votes
1answer
107 views

The representation of F1-score on the Precision-Recall Curve

Is there a way to project the F1-score on the precision-recall curve for a such binary classifier? Is there a relationship between the area under the precision-recall curve and F1-score? ...
0
votes
0answers
23 views

Understanding model performance

I built a multiclass (and very imbalanced classes) classifier. When evaluating I found an average F1 of .98 and the classifier seems to be working rather well. However, on evaluating the ROC and ...
0
votes
0answers
45 views

Best performance metric for highly imbalanced dataset f1 score vs kappa vs AUROC

I have highly imbalanced data (like fraud detection). I usually use f1 score to evaluate model performance. But I also saw people recommend AUROC and cohen's kappa. I'm seeking expert opinion on what ...
0
votes
2answers
33 views

Plot ROC or PR curves from either the X,Y coordinates (i.e TPR/TNR; or PPV/TPR) or list of predictions (class probabilities)? [closed]

I have a list of X,Y coordinates for plotting both a ROC curve and a PR curve. I also have the data which was used to calculate those coordinates (i.e. a list of individual predictions with binary ...
0
votes
0answers
29 views

which recall value to plot for same precision in PR curve?

Suppose, after sorting the true labels by the corresponding classifier scores, we obtain the following: $$[False, True, False, True, True, True, False, False],$$ which leads to the following points ...
1
vote
0answers
30 views

Computing recall on classification task

I have done google searches on computing "recall @ X" several times in the last few months, and every page I read gives me the same answer: $\frac{TP}{TP + FN}$ ... but the story I get from my PI and ...
0
votes
0answers
31 views

What a convex Precision-Recall curve means for training dataset?

Situation I have trained a GBDT model(gradient-boosted decision tree, a tree ensemble model) with a training dataset, and when I calculate PR curve on the same training set, it looks convex: For ...
0
votes
0answers
11 views

what happened if we reverse a fraud classification model?

Let us assume we train a classification model on fraud data sets to detect fraud and no fraud. The classification model threshold has high precision and low recall with ROC AUC=0.9, so we will only ...
0
votes
1answer
22 views

Precision and Recall: Valid Combination?

I'm looking at a precision-recall curve for a binary classification task. My precision-recall curve intersects the y-axis (precision) at 60% and the x-axis at 15%. So I get 15% precision at 100% ...
3
votes
0answers
689 views

Confidence interval of precision / recall and F1 score

To summarise the predictive power of a classifier for end users, I'm using some metrics. However, as the users input data themselves, the amount of data and class distribution varies a lot. So to ...
0
votes
1answer
867 views

Is there any difference between Sensitivity and Recall?

In most of the places, I have found that sensitivity=recall. In terms of the Confusion Matrix, the formula for both of these is the same: $TP/(TP+FN)$. Is there any difference between these two ...
1
vote
0answers
62 views

Poor P-R curve for binary classifier trained on balanced data, with imbalanced test data

I have a very imbalanced dataset (9:1), for which I have performed under-sampling and achieved a balanced training set (~130k samples total post balancing). I am performing classification using ...
0
votes
1answer
55 views

Interpreting precision and recall graphs

i have a CNN for sentiment analysis whose precision and recall for validation data over 10 training epochs is (average:macro): The dataset contains more positive samples than negatives.I have ...
1
vote
1answer
120 views

ROC and PR curves interpretation

Having two plots of ROC and PR curves (by scikit-learn) on one dataset raised me a question. The generated Precision-recall plot shows high precision and high recall, that is, low false positive rate ...
0
votes
0answers
13 views

Average Precision or FBeta & Decision Threshold Tuning for Binary Classifier [duplicate]

I'm working with an imbalanced binary classifier data set (3% positive) in sklearn. The cost of a false negative is extremely high so recall is much more important than precision. To baseline my ...
0
votes
0answers
18 views

Which curve comparison should I use to evaluate the performance of a recommender?

I am building a recommender system on the Last.FM dataset (link here) (1,892 users and 17,632 artists and the number of times a particular artist was listened to by a user). Next, the raw dataset was ...
2
votes
0answers
192 views

What does it mean if the ROC AUC is high and the Average Precision is low?

I have a model that produces a high ROC AUC (0.90), but at the same time a low average precision (0.30). From what I've found, I think it might have to do something with imbalanced data (which the ...
0
votes
1answer
23 views

Is there a 'precision' or 'recall' metric for True Negatives?

Noob question.. Accuracy is defined as percentage of predictions that are correct. Accuracy = (True Positives + True Negatives)/(All classifications) Precision is defined as percentage of ...
0
votes
0answers
29 views

Improving cold-start recommend systems

I am using tensorrec (python framework for recommendation, based on tensorflow) to predict a users choice of content, based on the users meta-data. My current accuracy is at about 2,5% *. Since this ...