4
$\begingroup$

I have a set of users. A classification algorithm is applied on all users, and I take (call analyzedExperts) a set of users which are binary classified (expert & non-expert).

And I use another method to evaluate this algorithm. That does the same thing, and I take another set of users which are also binary classified (call realExperts).

But if I want to measure the precision and recall, I take the same result for both. The sets analyzedExperts and realExperts have both the same size of data.

I don't understand why they are same, and don't know whether it is normal. P.S. I'm not sure whether the precision and recall is a good way to measure the evaluating the results.

EDIT:

Thus, the question is: if they are equalsized, precision and recall have to be same?

Suppose they have 3 users in common (True positive). FN and FP will be always same because they both have the same size. What implicits that the precision and recall will be same.

Second question might be then: does realExperts has to have greater size? Or is it not the good place to use precision or recall?

enter image description here

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ What do you mena with "I take the same results for both"? On which basis do you calculate recall and precision? Do you compare the numbers of precision and recall for one given set, or do you compare the precision of two sets (and they are equal), and you compare the recall of two sets (and they are equal)? $\endgroup$ – Roland May 12 '14 at 19:24
  • $\begingroup$ No, based on realExperts, i take (just one) precision and recall value of analyzedExperts. And i take precision value which is same with recall value. $\endgroup$ – Asqan May 12 '14 at 19:52
3
$\begingroup$

Let's call the number of users who are correctly classified as experts by $tp$ (true positive), the number of users who are incorrectly classified as non-experts (but they are experts) by $fn$ (false negative), and the number who are incorrectly classified as experts (because they are not) by $fp$ (false positive).

The precision is defined as $p = \frac{tp}{tp + fp}$, where the recall is defined as $r = \frac{tp}{tp + fn}$. If precision and recall are equal, we have $p=r$, and since they have the same denominator, we get $fp = fn$.

This means that our algorithm has classified an equal amount of users as false positives, as it classified false negatives. This may be a good thing if the data set had an equal set of experts/non-experts, but it also may be a hint of the fact that too many non-experts were classified as experts (if there were not many experts in the set), or too many experts were not classified as experts (of there were many experts in the set).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Until the last paragraph, i agree with you. But the reason is different than you explained in the last paragraph, i think. I have updated my question with extra clarification. I think thats the reason but i dont know whether it is a good way to apply precision&recall metrics and whether i'm doing something wrong $\endgroup$ – Asqan May 12 '14 at 22:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.