I want to find the mean average precision (meanAP) from a classification problem. I am using liblinear for classification and I am trying to use vlfeat for the precision because it already includes a built-in function to compute precision.
From liblinear I run the command
[predicted_label, accuracy, prob_estimates] = predict(testing_label_vector, testing_instance_matrix, model);
prob_estimates matrix has each row being a separate sample, and each column being the score of it matching that class.
Given this I wrote a short function to compute the meanAP. The inputs are prob, which is the same
prob_estimates from liblinear; and
allTrueClass which is the true class of the testing samples.
function meanAP = computeMeanAP(prob,allTrueClass) classType = unique(allTrueClass); % for each class for i=1:numel(classType) % get the "true class" to be 1 for that class and -1 for everthing else labels = allTrueClass == classType(i); labels = 2*labels -1; % then get the probability for that class scores = prob(:,i); % use vlfeat to get the average ap [rc, pr, info] = vl_pr(labels, scores) ; ap(i) = info.ap_interp_11; end % return meanAP meanAP = mean(ap); end
This function uses the vl_pr function from vlfeat to compute the average precision for a single class. And then I average the results over all classes.
However with this code I get some strange results. If I randomly change the order of the training samples, I get the same classification accuracy but a very different meanAP. Can anyone figure out where I am going wrong here?