I want to compute Mean Average Precision (MAP) from Average Precision (AP) over all users and over every 5 minute intervals of a day. How do I compute MAP from AP for both these dimensions [user, time interval].
That is I get an AP per user, per 5 minute. I want MAP for whole day averaged across all users.
Is there a "MMAP" (for lack of better word)? Does it even make sense as I am computing average over an average?
For any given period, sum over the Average Precision over all queries in that period and divide by the total number of queries to give the Mean Average Precision for that period. I'd advise against doing a "MMAP" as this will essentially be a weighted average (by which I mean don't calculate MAP for the day by averaging the MAP's taken over discrete intervals throughout the day). Instead, multiply each individual MAP by the corresponding number of queries it represents, sum all of these terms together, and then divide again by the total number of queries for the whole day to give the unbiased MAP from the precalculated MAPs (allowing you to take advantage of your calculated summaries instead of summing over everything each time).
Really, the easiest thing to do is probably to just tally AP and number of queries throughout the day as a running sum, and then divide the two to give you the daily MAP for all users.
Disclaimer: I'm referencing this interpretation of MAP.
EDIT: Allow me to make explicit why sum(MAP)/n is actually the weighted average calculation.
You are calculating MAP at five minute intervals. The MAP formula is (according to wikipedia):
$\operatorname{MAP} = \frac{\sum_{q=1}^Q \operatorname{AveP(q)}}{Q}$
Where $Q$ is the number of queries and $\operatorname{AveP(q)}$ is the average precision for a single query.
Let's consider two five minute intervals, $A$ and $B$. Let's say $Q_A = 2$ and $Q_B=100$. Further, let's pretend that all of the queries in $A$ had $\operatorname{AveP(q)} = 0.5$ and all the queries in $B$ had $\operatorname{AveP(q)} = 0.001$. Therefore $MAP_A = 0.5$ and $MAP_B = 0.001$. If we take the mean of these two MAPs, we'll get $MAP'_{A,B} = \frac{0.5 + 0.001}{2} = .2505$.
The problem here is that the $B$ interval represents 100 queries and the A interval only represents 2, so intuitively we know that the true mean average precision for all queries should be much closer to $MAP_B$ than we're observing. The true value for $MAP_{A,B}$ is:
$ \begin{align} MAP_{A,B} &= \frac{\sum_{q=1}^{Q_{A,B}} \operatorname{AveP(q)}}{Q_{A,B}} \\\ &= \frac{\sum_{q=1}^{Q_{A}} \operatorname{AveP(q)} + \sum_{q=1}^{Q_{B}} \operatorname{AveP(q)}}{Q_{A}+Q_{B}} \\\ &=\frac{2*0.5 + 100*0.001}{2+100} \\\ &=\frac{1.1}{102} \\\ &= 0.0108 \end{align} $
Taking the mean of MAPs calculated on set intervals but representing different numbers of queries significantly biases the calculation towards intervals that had few queries. But really, you want intervals that had lots of queries to dominate your calculation.
I was suggesting that, since you already have MAP values calculated, you could generate an unbiased average by doing:
$MAP_{1,2, ... N} = \frac{\sum_{i=1}^N Q_i * MAP_i}{\sum_{i=1}^N Q_i} = \frac{\sum_{i=1}^N AP_i}{\sum_{i=1}^N Q_i}$
This way you recover the respective average precision sums in the numerator and divide out by the true number of queries for the entire set to calculate your MAP for the entire time range.
