Skip to main content

Questions tagged [average-precision]

For questions related to the Average Precision metric.

Filter by
Sorted by
Tagged with
1 vote
1 answer
15 views

How does average_precision_score metric in scikit-learn work for non-probability prediction scores

Scikit-learn has an AP metric function Here The description of y_score (predictions) says :- ...
Anmol's user avatar
  • 113
0 votes
0 answers
64 views

Why regarding mean Average Precision for 50 and for 50 to 95? And can you average different evaluations?

I saw a lot of papers using mAP@50 and mAP0:5:95, but in terms of deciding how precise a trained object detection model is, I do not understand why often both values are discussed and shown. Wouldn't ...
Taka Incur's user avatar
2 votes
0 answers
154 views

Choosing a metric for lightGBM classifier (mean average precision k)

I have developed a binary classifier using LightGBM, where I've primarily used the AUC metric due to its simplicity, ease of use, and interpretability. Recently, I've taken an interest in utilizing ...
Programming Noob's user avatar
1 vote
1 answer
316 views

Why use average_precision_score from sklearn? [duplicate]

I have precision and recall values and want to measure an estimator performance: ...
Ars ML's user avatar
  • 31
0 votes
0 answers
39 views

Methods to approximate Area under Precision-Recall Curve

average_precision_score from sklearn uses formula: ap = sum( (recall[k+1] - recall[k]) * precision[k+1] ) But trapezoidal rule implies: ...
Ars ML's user avatar
  • 31
0 votes
0 answers
107 views

How to calculate Mean Average Precision at a fixed IoU threshold?

Papers such as COCO and the VisDrone often list MAP, MAP50, MAP75 in their evaluation criteria. To my understanding, MAP is the averaged AP for all classes across an IoU range (0.5 - 0.95 for example),...
Dani Mazahreh's user avatar
0 votes
0 answers
31 views

Mean AUC vs Mean Average Precision?

Both AUC and Average Precision (AP) are commonly used to evaluate the quality of a ranked list of relevant / irrelevant documents. When evaluating a search engine, we want to evaluate the quality over ...
usual me's user avatar
  • 1,247
2 votes
0 answers
36 views

Is there a way to effect the shape of precision-recall curve?

As long as I know, for both ROC and PR curves, the classifier performance is usually measured by the AUC. This might indicate that classifiers with equivalent performance might have different ROC/PR ...
Gideon Kogan's user avatar
1 vote
0 answers
41 views

Regarding area ABOVE the curve - complement of AUROC

When handling probabilities close to 1, it is often more helpful to use the complement (i.e. 1-P). For instance, we say "there is a 1 in 1,000,000 chance of an event occurring", instead of &...
Cyruno's user avatar
  • 13
0 votes
0 answers
332 views

Comparing AUC-PR between groups with different baselines

So I know that the area under the precision-recall curve is often a more useful metric than AUROC when dealing with highly imbalanced datasets. However, while AUROC can easily be used to compare ...
Eike P.'s user avatar
  • 3,068
0 votes
1 answer
39 views

Better in AUC and AUC PR, but lower in the optimal threshold

Suppose we have two models; model A and model B. Model A outperforms both AUC ROC and AUC PR to model B. However, when we compare the two models with their optimal threshold values, model B ...
R and C.F's user avatar
1 vote
0 answers
26 views

Does a model with 0.5 AUROC imply an average precision equal to the proportion of positive examples?

A random model has an area under the ROC curve equal to 0.5. We also know that a random model has an area under the Precision-Recall curve equal to the proportion (p) of positive examples. Then, here'...
killezio's user avatar
  • 111
0 votes
0 answers
31 views

Average Margin of Error

Sampling 100 random respondents for a binary true/false response from a total population of 220,000,000 yields a margin of error of 9.8%. If a new random sample of 100 respondents from the same ...
B Chase's user avatar
1 vote
0 answers
102 views

roc-auc≈0.5, accuracy≈precision≈average_precision≈65%, recall≈1 [closed]

After reading this and this , I tried it on mine by fitting the 2-input model i.e. text and numerical. The result remains similar even several attempts on tuning the hyperparameters e.g. embedding ...
user357565's user avatar
1 vote
0 answers
80 views

Questions about mAP results from YOLOv2 paper

In the YOLOv2 paper, is the mAP metric displayed in Table 3 calculated in the same way as the metric in the column '0.5' from Table 5?
Yandle's user avatar
  • 1,189
1 vote
0 answers
179 views

Can you estimate average precision from log loss?

I am doing my final thesis in the field of Deepfakes and their detection. The final outcome is to have a binary classifier which could predict which video was updated and which was not. In other words,...
MichiganMagician's user avatar
0 votes
1 answer
1k views

What is the average precision in the case of no positives for a given category in the context of object detection

In attempting to calculate the average precision of an object detection model, I am wondering about an edge case. Suppose at evaluation time that for a given category, that no detections of that ...
IntegrateThis's user avatar
9 votes
1 answer
8k views

Sklearn Average_Precision_Score vs. AUC

Can someone explain in an intuitive way the difference between Average_Precision_Score and AUC? I read the documentation and understand that they are calculated slightly differently. But what is the ...
Windstorm1981's user avatar
1 vote
0 answers
1k views

XGBoost Mean Average Precision eval_metric for Classification

I am testing XGBoostClassifier for a binary classification problem. I have tried a few base models, done some simple parameter tuning, and performed feature selection using sklearn's ...
Jake Niederer's user avatar
1 vote
0 answers
2k views

how to find mean average precision of object detection algorithms

To start with, I would like to mention another question which was asked in a better way. But my problem differs. pseudo code for the algorithms I have four different object detection algorithms which ...
Qasim Ali's user avatar
  • 111
1 vote
1 answer
212 views

How can mAP be less than all of mAP_S, mAP_M, and mAP_L?

I was looking at this graph from Learning Data Augmentation Strategies for Object Detection and I noticed that the value for mAP is lower than all of mAP_S, mAP_M, and mAP_L for the third set of ...
jss367's user avatar
  • 408
7 votes
2 answers
1k views

Is it better to compute Average Precision using the trapezoidal rule or the rectangle method?

Background Average precision is a popular and important performance metric widely used for, e.g., retrieval and detection tasks. It measures the area under the precision-recall curve, which plots the ...
Callidior's user avatar
0 votes
0 answers
136 views

Acceptable level of mAP in computer vision applied to health applications

EDIT: This question is meant for those who previously understand what mAP means but for contextualizing this question properly, it is the mean average precision as defined by Microsoft COCO i.e. the ...
Matias Haeussler's user avatar
1 vote
0 answers
31 views

Calculating three average precisions and a single value for ROC from raw predicted class outputs

I'm not a statistician or mathematician so I apologize if I use any terms incorrectly. Please do point out any errors in my use of terminology. The four values I need are the equivalent of Weka's ROC ...
Doc Octal's user avatar
  • 131
1 vote
1 answer
535 views

performance measure suited for imbalanced classes and robust towards changing class ratios

I am looking for the best performance measure. My use case: I want to find out which dataset can be modelled best with binary classification. The datasets have an active minority class I am ...
user954923's user avatar
2 votes
1 answer
239 views

Confusion about computation of average precision

I am trying to learn what AP (average precision) means and I came across this page: https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52 Here is the given formula: ...
user5054's user avatar
  • 1,559
0 votes
1 answer
173 views

How to calculate ± deviation for precision and recall?

I found a precision and recall report table like as below Precision Recall .470±.009 .934±.013 .239±.010 .610±.013 I need the guidelines for ±.009 and ±.013 ...
user1999109's user avatar
3 votes
1 answer
2k views

Calculating sklearn's average precision by hand

I'm trying to understand how sklearn's average_precision metric works. The reason I want to compute this by hand is to understand the details better, and to figure ...
StatsSorceress's user avatar
1 vote
0 answers
164 views

Is it normal to have higher cross validation score IN EACH ITERATION than testing score

I'm using 10 fold stratified cross validation, training:testing is 0.75:0.25. Balanced data. I'm using cross validation when doing feature selection with the 0.75 training data. The score I'm using is ...
Cherry Wu's user avatar
  • 331
2 votes
0 answers
692 views

Computing average precision metric and cost function for object detection task using scikitLearn and Tensorflow

I have a Data set that contains 5 thousand pictures of my object of interest and 5 thousand pictures with out it. I trained a Convolutional Neural Network using Tensor Flow to detect the position of ...
Manuel Sebastian Rios's user avatar
4 votes
1 answer
658 views

Intuitive or quantitative explanation of why we care about mean average precision (mAP) for CNN classifiers?

Consider CNN classifiers applied to some image classification tasks: to fix ideas, let's consider the ImageNet Challenge, where each image belongs to 1 of 1000 nonoverlapping classes, even though the ...
DeltaIV's user avatar
  • 18.1k
2 votes
1 answer
598 views

mAP calculation in object detection

I'm quite confused as to how I can calculate the AP (average precision) or mAP (mean average precision) to evaluate an object detection model. I specifically want to know if the True Positives (TP) ...
Desmanda's user avatar
2 votes
1 answer
359 views

Average precision when no relevant documents are found

I am building an algorithm that attempts to return relevant documents. If the query retrieves 10 documents but none are relevant how is the average precision calculated? Applying the AveP formula, it ...
Kevin's user avatar
  • 55
10 votes
2 answers
13k views

What does it mean if the ROC AUC is high and the Average Precision is low?

I have a model that produces a high ROC AUC (0.90), but at the same time a low average precision (0.30). From what I've found, I think it might have to do something with imbalanced data (which the ...
Icyeval's user avatar
  • 535
3 votes
1 answer
407 views

Evaluating rare event risk metrics

Suppose there is a rare event that happens on 3-7 days a year, and we are interested to predict days when it happens. We have two metrics, A and B, that both take values on onterval (0, 1) for any ...
kreo's user avatar
  • 153
2 votes
0 answers
1k views

How to count precision and recall for multiclass classification which returns top-5 classes per test example

This is how testing looks like: There is 100 target classes The test set consists of 10K elements - each one of them is tagged by one target class The distribution of classes over test set is ...
Ziemo's user avatar
  • 161
3 votes
1 answer
2k views

Using micro average vs. macro average vs. normal versions of precision and recall for a binary classifier

I have a logistic regression recommender model built on my data where I tried to predict one of two outcomes for each row. Let's call them success and ...
NeonBlueHair's user avatar
1 vote
1 answer
435 views

How can the mAP metric be meaningful for non-exhaustively labeled datasets (such as YouTube BoundingBox)?

I am interested in reproducing the object detection results found in the whitepaper describing the YouTube BoundingBox dataset (https://arxiv.org/pdf/1702.00824.pdf). What I don't understand is how ...
Kantthpel's user avatar
3 votes
0 answers
1k views

How to compute a precision-recall curve for an instance segmentation algorithm?

Having currently read some papers about proposed solutions to the problem of instance segmentation in images, (i.e. an algorithm that takes as input raw images, and outputs instance-wise segmentation ...
JLagana's user avatar
  • 306
16 votes
4 answers
29k views

Average Precision in Object Detection

I'm quite confused as to how I can calculate the AP or mAP values as there seem to be quite a few different methods. I specifically want to get the AP/mAP values for object detection. All I know for ...
User1915's user avatar
  • 461
1 vote
1 answer
1k views

Is an Average Precision of 60% acceptable output in a fraud detection machine learning algorithm? What does it signifies?

First question here, I am new to machine learning and wanted to understand the following: I used decision trees, boosting to classify fraud users and I am getting average precision around 60% on my ...
Jaideep Poonia's user avatar
2 votes
1 answer
3k views

How to calculate mean average precision given precision and recall for each class?

I use Pascal VOCdevkit to calculate object detection average precision for each class, but how can I get mean average precision for the whole dataset? Should I average each average precision or should ...
SunshineAtNoon's user avatar
0 votes
1 answer
354 views

Absolute Average deviation in percentage calculation

Sorry if my terminology is incorrect. I am trying to calculate the average error of prediction to be represented in percentage. For example, I should be able to say, the predicted values are on ...
Ahmadov's user avatar
  • 113
4 votes
0 answers
2k views

Calculating the standard deviation of the mean of average rates of speed

Is it possible to determine the mean value of a point by averaging the average rate of ranges that contain that point, and if so, how can the uncertainty of that value be accurately determined? I ...
Nick Anderegg's user avatar
1 vote
1 answer
2k views

determine the mean cost per unit

I have N orders. Each order consist of x units and I know the total cost of each order. If I would like to determine the mean cost per unit do I use approach 1 or 2 listed as follows: (1) Determine ...
cs0815's user avatar
  • 2,245
57 votes
2 answers
77k views

Area under Precision-Recall Curve (AUC of PR-curve) and Average Precision (AP)

Is Average Precision (AP) the Area under Precision-Recall Curve (AUC of PR-curve) ? EDIT: here is some comment about difference in PR AUC and AP. The AUC is obtained by trapezoidal interpolation ...
mrgloom's user avatar
  • 2,207
4 votes
1 answer
2k views

Average precision when not all the relevant documents are found

I can't find on the Internet a proper source that explains this. I have built a search engine that for a particular query retrieves 5 relevant document out of the 10 relevant documents. When I ...
ramborambo's user avatar
0 votes
0 answers
96 views

Obtain Precision and Recall from Click through data

I am trying to build a graph of precision and recall using click data. I have two data sources. First data source has all the user clicked item_ids based on a given query_id. Second data source has ...
add-semi-colons's user avatar
0 votes
2 answers
935 views

Statistical significance test for multiple binary classification problems

Let $C_1$, $C_2$ be two binary classifiers, which are used to classify some data (images, videos, etc) to $30$ different classes, using an one-against-all approach. Then, we have two $30$-dimensional ...
nullgeppetto's user avatar
33 votes
1 answer
33k views

Mean Average Precision vs Mean Reciprocal Rank

I am trying to understand when it is appropriate to use the MAP and when MRR should be used. I found this presentation that states that MRR is best utilised when the number of relevant results is less ...
K G's user avatar
  • 431