# Mean Average Precision (MAP) in two dimensions

I want to compute Mean Average Precision (MAP) from Average Precision (AP) over all users and over every 5 minute intervals of a day. How do I compute MAP from AP for both these dimensions [user, time interval].

That is I get an AP per user, per 5 minute. I want MAP for whole day averaged across all users.

Is there a "MMAP" (for lack of better word)? Does it even make sense as I am computing average over an average?

• Please edit this question to tell us what you mean by "AP" and "MAP."
– whuber
Feb 10, 2014 at 22:46
• Sorry - Mean Average Precision and Average Precision. Feb 10, 2014 at 22:54

For any given period, sum over the Average Precision over all queries in that period and divide by the total number of queries to give the Mean Average Precision for that period. I'd advise against doing a "MMAP" as this will essentially be a weighted average (by which I mean don't calculate MAP for the day by averaging the MAP's taken over discrete intervals throughout the day). Instead, multiply each individual MAP by the corresponding number of queries it represents, sum all of these terms together, and then divide again by the total number of queries for the whole day to give the unbiased MAP from the precalculated MAPs (allowing you to take advantage of your calculated summaries instead of summing over everything each time).

Really, the easiest thing to do is probably to just tally AP and number of queries throughout the day as a running sum, and then divide the two to give you the daily MAP for all users.

Disclaimer: I'm referencing this interpretation of MAP.

EDIT: Allow me to make explicit why sum(MAP)/n is actually the weighted average calculation.

You are calculating MAP at five minute intervals. The MAP formula is (according to wikipedia):

$\operatorname{MAP} = \frac{\sum_{q=1}^Q \operatorname{AveP(q)}}{Q}$

Where $Q$ is the number of queries and $\operatorname{AveP(q)}$ is the average precision for a single query.

Let's consider two five minute intervals, $A$ and $B$. Let's say $Q_A = 2$ and $Q_B=100$. Further, let's pretend that all of the queries in $A$ had $\operatorname{AveP(q)} = 0.5$ and all the queries in $B$ had $\operatorname{AveP(q)} = 0.001$. Therefore $MAP_A = 0.5$ and $MAP_B = 0.001$. If we take the mean of these two MAPs, we'll get $MAP'_{A,B} = \frac{0.5 + 0.001}{2} = .2505$.

The problem here is that the $B$ interval represents 100 queries and the A interval only represents 2, so intuitively we know that the true mean average precision for all queries should be much closer to $MAP_B$ than we're observing. The true value for $MAP_{A,B}$ is:

\begin{align} MAP_{A,B} &= \frac{\sum_{q=1}^{Q_{A,B}} \operatorname{AveP(q)}}{Q_{A,B}} \\\ &= \frac{\sum_{q=1}^{Q_{A}} \operatorname{AveP(q)} + \sum_{q=1}^{Q_{B}} \operatorname{AveP(q)}}{Q_{A}+Q_{B}} \\\ &=\frac{2*0.5 + 100*0.001}{2+100} \\\ &=\frac{1.1}{102} \\\ &= 0.0108 \end{align}

Taking the mean of MAPs calculated on set intervals but representing different numbers of queries significantly biases the calculation towards intervals that had few queries. But really, you want intervals that had lots of queries to dominate your calculation.

I was suggesting that, since you already have MAP values calculated, you could generate an unbiased average by doing:

$MAP_{1,2, ... N} = \frac{\sum_{i=1}^N Q_i * MAP_i}{\sum_{i=1}^N Q_i} = \frac{\sum_{i=1}^N AP_i}{\sum_{i=1}^N Q_i}$

This way you recover the respective average precision sums in the numerator and divide out by the true number of queries for the entire set to calculate your MAP for the entire time range.

• So in summary you are saying do a weighted average rather than average - makes sense. Feb 17, 2014 at 18:42
• Actually, I'm suggesting the opposite. The method I'm recommending gives you an unweighted (i.e. unbiased) average. I was pointing out that the method you had originally suggested would unintentionally produce a weighted average, but with arbitrary weights. Feb 17, 2014 at 19:47
• hmmm - "multiply each individual MAP by the corresponding number of queries it represents, sum all of these terms together, and then divide again by the total number of queries" - sounds more like weighted average to me instead of plain average - sum(MAP)/n! Feb 19, 2014 at 0:49
• I just added a detailed explanation to my answer of why your intuition is not correct. $\sum MAP / n$ actually gives you a weighted average. Feb 19, 2014 at 18:16