Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [average-precision]

The tag has no usage guidance.

1
vote
1answer
21 views

Confusion about computation of average precision

I am trying to learn what AP (average precision) means and I came across this page: https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52 Here is the given formula: ...
0
votes
1answer
20 views

How to calculate ± deviation for precision and recall?

I found a precision and recall report table like as below Precision Recall .470±.009 .934±.013 .239±.010 .610±.013 I need the guidelines for ±.009 and ±.013 ...
0
votes
0answers
26 views

Calculating sklearn's average precision by hand

I'm trying to understand how sklearn's average_precision metric works. The reason I want to compute this by hand is to understand the details better, and to figure ...
0
votes
0answers
11 views

What values should the Interpolated 11-point precision curve, measured at k, have when P@k = 0?

I'm evaluating a few retrieval models using the 11-point Interpolated precision x recall curve (here is a description of the protocol I've been using: https://nlp.stanford.edu/IR-book/html/htmledition/...
1
vote
0answers
40 views

Is it normal to have higher cross validation score IN EACH ITERATION than testing score

I'm using 10 fold stratified cross validation, training:testing is 0.75:0.25. Balanced data. I'm using cross validation when doing feature selection with the 0.75 training data. The score I'm using is ...
0
votes
0answers
247 views

Expected average precision for random ranked list

Suppose we are evaluating the label ranking average precision (LRAP), which is an evaluation metric used to compare a ranked list to a ground truth (binary) label vector. Background and Definition ...
1
vote
0answers
229 views

Computing average precision metric and cost function for object detection task using scikitLearn and Tensorflow

I have a Data set that contains 5 thousand pictures of my object of interest and 5 thousand pictures with out it. I trained a Convolutional Neural Network using Tensor Flow to detect the position of ...
4
votes
1answer
90 views

Intuitive or quantitative explanation of why we care about mean average precision (mAP) for CNN classifiers?

Consider CNN classifiers applied to some image classification tasks: to fix ideas, let's consider the ImageNet Challenge, where each image belongs to 1 of 1000 nonoverlapping classes, even though the ...
0
votes
1answer
222 views

mAP calculation in object detection

I'm quite confused as to how I can calculate the AP (average precision) or mAP (mean average precision) to evaluate an object detection model. I specifically want to know if the True Positives (TP) ...
1
vote
1answer
72 views

Average precision when no relevant documents are found

I am building an algorithm that attempts to return relevant documents. If the query retrieves 10 documents but none are relevant how is the average precision calculated? Applying the AveP formula, it ...
2
votes
0answers
261 views

What does it mean if the ROC AUC is high and the Average Precision is low?

I have a model that produces a high ROC AUC (0.90), but at the same time a low average precision (0.30). From what I've found, I think it might have to do something with imbalanced data (which the ...
0
votes
0answers
10 views

Value of averages with high standard deviation

When building bus time tables and looking at performance we often look at running times vs scheduled times. The data sets used include hundreds of trips ranging between 45 - 90 minutes scheduled time. ...
2
votes
1answer
78 views

Evaluating rare event risk metrics

Suppose there is a rare event that happens on 3-7 days a year, and we are interested to predict days when it happens. We have two metrics, A and B, that both take values on onterval (0, 1) for any ...
2
votes
0answers
1k views

How to count precision and recall for multiclass classification which returns top-5 classes per test example

This is how testing looks like: There is 100 target classes The test set consists of 10K elements - each one of them is tagged by one target class The distribution of classes over test set is ...
2
votes
1answer
859 views

Using micro average vs. macro average vs. normal versions of precision and recall for a binary classifier

I have a logistic regression recommender model built on my data where I tried to predict one of two outcomes for each row. Let's call them success and ...
1
vote
1answer
371 views

How can the mAP metric be meaningful for non-exhaustively labeled datasets (such as YouTube BoundingBox)?

I am interested in reproducing the object detection results found in the whitepaper describing the YouTube BoundingBox dataset (https://arxiv.org/pdf/1702.00824.pdf). What I don't understand is how ...
2
votes
0answers
752 views

How to compute a precision-recall curve for an instance segmentation algorithm?

Having currently read some papers about proposed solutions to the problem of instance segmentation in images, (i.e. an algorithm that takes as input raw images, and outputs instance-wise segmentation ...
8
votes
4answers
17k views

Average Precision in Object Detection

I'm quite confused as to how I can calculate the AP or mAP values as there seem to be quite a few different methods. I specifically want to get the AP/mAP values for object detection. All I know for ...
1
vote
1answer
552 views

Is an Average Precision of 60% acceptable output in a fraud detection machine learning algorithm? What does it signifies?

First question here, I am new to machine learning and wanted to understand the following: I used decision trees, boosting to classify fraud users and I am getting average precision around 60% on my ...
1
vote
1answer
3k views

How to calculate mean average precision given precision and recall for each class?

I use Pascal VOCdevkit to calculate object detection average precision for each class, but how can I get mean average precision for the whole dataset? Should I average each average precision or should ...
0
votes
1answer
157 views

Absolute Average deviation in percentage calculation

Sorry if my terminology is incorrect. I am trying to calculate the average error of prediction to be represented in percentage. For example, I should be able to say, the predicted values are on ...
3
votes
0answers
974 views

Calculating the standard deviation of the mean of average rates of speed

Is it possible to determine the mean value of a point by averaging the average rate of ranges that contain that point, and if so, how can the uncertainty of that value be accurately determined? I ...
0
votes
1answer
842 views

determine the mean cost per unit

I have N orders. Each order consist of x units and I know the total cost of each order. If I would like to determine the mean cost per unit do I use approach 1 or 2 listed as follows: (1) Determine ...
22
votes
2answers
24k views

Area under Precision-Recall Curve (AUC of PR-curve) and Average Precision (AP)

Is Average Precision (AP) the Area under Precision-Recall Curve (AUC of PR-curve) ? EDIT: here is some comment about difference in PR AUC and AP. The AUC is obtained by trapezoidal interpolation ...
2
votes
1answer
1k views

Average precision when not all the relevant documents are found

I can't find on the Internet a proper source that explains this. I have built a search engine that for a particular query retrieves 5 relevant document out of the 10 relevant documents. When I ...
0
votes
0answers
88 views

Obtain Precision and Recall from Click through data

I am trying to build a graph of precision and recall using click data. I have two data sources. First data source has all the user clicked item_ids based on a given query_id. Second data source has ...
0
votes
2answers
510 views

Statistical significance test for multiple binary classification problems

Let $C_1$, $C_2$ be two binary classifiers, which are used to classify some data (images, videos, etc) to $30$ different classes, using an one-against-all approach. Then, we have two $30$-dimensional ...
9
votes
1answer
10k views

Mean Average Precision vs Mean Reciprocal Rank

I am trying to understand when it is appropriate to use the MAP and when MRR should be used. I found this presentation that states that MRR is best utilised when the number of relevant results is less ...
7
votes
0answers
357 views

Baseline for Precision-Related Metrics

When working with ROC-AUC as a metric for binary classification, one often considers a value of 0.5 as a baseline from a random classifier (i.e. a data-blind classifier that randomly classifies test ...
11
votes
2answers
2k views

How to compare two ranking algorithms?

I want to compare two ranking algorithms. In these algorithms, client specifies some conditions in his/her search. According to the client`s requirements, these algorithm should assign a score for ...
2
votes
0answers
83 views

Average of percentages / prove causal relationship between sale dates & margin sales

I just wanted to make sure I'm right here. I have a situation where I need to prove that on days where we sell more than 10 items, our margin (sale price vs. suggested price) goes up. I'm not really ...
3
votes
1answer
6k views

“Mean average precision” (MAP) evaluation statistic - understanding good/bad/chance values

I'm evaluating a multilabel classifier. I'm familiar with the Area Under the Curve statistic, which has some nice properties (e.g. chance level is always 50%). But for some applications, it's more ...
0
votes
1answer
695 views

Mean Average Precision (MAP) in two dimensions

I want to compute Mean Average Precision (MAP) from Average Precision (AP) over all users and over every 5 minute intervals of a day. How do I compute MAP from AP for both these dimensions [user, time ...
2
votes
1answer
270 views

How can I check if two numbers are equal (with some allowed error)?

I have a lot (about 10000 or more) measurement results. I measured the performance of different algorithms (it doesnt really matter which algorithms for now on). I want to check if my measurement ...
3
votes
0answers
2k views

Averaged vs. Combined k-fold cross validation and leave-one-out

calculating recall/precision from k-fold cross validation (or leave-one-out) can be performed either by averaging the recall/precision values obtained from different k folds or by combining the ...
4
votes
2answers
3k views

Derive percentiles from binned data

The question below was asked on a sister site (Stack Overflow) back in 2010 by a user still active there (to me it seems more suitable here, for example quite similar to 21422): I have a bunch of ...
2
votes
1answer
495 views

Precision, Recall and area under ROC curve as sample size increases

The following is a question from an exam paper on evaluating the performance of search engines. To this day I looked in my text book and literally close to 50 web pages and I can't find one convincing ...