I can't find on the Internet a proper source that explains this.

I have built a search engine that for a particular query retrieves 5 relevant document out of the 10 relevant documents.

When I calculate the average precision, I sum the Precision@k, where k is relevant. At this stage, should I divide by 10 or by 5?


1 Answer 1


Average Precision should be divided by 10 and not 5.

Formula from Manning's Introduction to Information Retrieval

$$ MeanAveragePrecision(Q) = \frac{1}{|Q|} \sum_{j=1}^{|Q|}\frac{1}{m_j} \sum_{k=1}^{m_j} Precision(R_{jk}) $$

for query $q_j \in Q$ containing relevance documents ${d_1, ..., d_{m_j}}$ and $R_{jk}$ is the set of ranked results until document $d_k$. You are only looking for the Average Precision which is the inner sum. You can see that $\sum Precision$ divides by $m_j$ which is the total number of relevant documents.

Wikipedia's Average Precision page has the formula

$$ AveragePrecision = \frac {\sum_{k=1}^{n} P(k) \times rel(k)}{numberOfRelevantDocuments} $$ where $P(k)$ is precision@k and $rel(k)$ is an indicator function equaling 1 if the item at rank k is a relevant document, zero otherwise. It also says the "is over all relevant documents and the relevant documents not retrieved get a precision score of zero"

  • $\begingroup$ I have seen a few example on university solutions to homeworks and this has been pretty inconsistent. Is there any other source different from wikipedia? $\endgroup$
    – ramborambo
    May 10, 2015 at 16:06
  • $\begingroup$ @ramborambo I believe Manning's definition and Wikipedia's definition are equivalent. Are they not? $\endgroup$
    – Eric Farng
    May 10, 2015 at 20:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.