1
$\begingroup$

I want to find the mean average precision (meanAP) from a classification problem. I am using liblinear for classification and I am trying to use vlfeat for the precision because it already includes a built-in function to compute precision.

From liblinear I run the command

[predicted_label, accuracy, prob_estimates] = predict(testing_label_vector, testing_instance_matrix, model);

The prob_estimates matrix has each row being a separate sample, and each column being the score of it matching that class.

Given this I wrote a short function to compute the meanAP. The inputs are prob, which is the same prob_estimates from liblinear; and allTrueClass which is the true class of the testing samples.

function meanAP = computeMeanAP(prob,allTrueClass)
classType = unique(allTrueClass);

% for each class
for i=1:numel(classType)
    % get the "true class" to be 1 for that class and -1 for everthing else
    labels = allTrueClass == classType(i);
    labels = 2*labels -1;

    % then get the probability for that class
    scores = prob(:,i);

    % use vlfeat to get the average ap
    [rc, pr, info] = vl_pr(labels, scores) ;

    ap(i) = info.ap_interp_11;
end

% return meanAP
meanAP = mean(ap);

end

This function uses the vl_pr function from vlfeat to compute the average precision for a single class. And then I average the results over all classes.

However with this code I get some strange results. If I randomly change the order of the training samples, I get the same classification accuracy but a very different meanAP. Can anyone figure out where I am going wrong here?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.