Skip to main content
corrected a typo related to the meaning of precision
Source Link
Michael R. Chernick
  • 43.2k
  • 28
  • 85
  • 159

The definition of "optimal" will of course depend on your specific goals, but here are a few relatively "standard" methods:

  • Equal error rate (EER) point: the point where precision equals recall. This feels to some people like a "natural" operating point.

  • A refined and more principled version of the above is to specify cost of the different kind of errors and optimize that cost. Say misclassifying an item (an error in precision) is twice as expensive as missing an item completely (error in recall). Then the best operating point is that where (1 - recall) = 2*(1 - precision).

  • In some problems people have a natural minimal acceptable rate of either precision or recall. Say you know that if more than 20% of retrieved data is incorrect, the users will stop using your application. Then it is natural to set precision to 80% (or a bit higherlower) and accept whatever recall you have at that point.

The definition of "optimal" will of course depend on your specific goals, but here are a few relatively "standard" methods:

  • Equal error rate (EER) point: the point where precision equals recall. This feels to some people like a "natural" operating point.

  • A refined and more principled version of the above is to specify cost of the different kind of errors and optimize that cost. Say misclassifying an item (an error in precision) is twice as expensive as missing an item completely (error in recall). Then the best operating point is that where (1 - recall) = 2*(1 - precision).

  • In some problems people have a natural minimal acceptable rate of either precision or recall. Say you know that if more than 20% of retrieved data is incorrect, the users will stop using your application. Then it is natural to set precision to 80% (or a bit higher) and accept whatever recall you have at that point.

The definition of "optimal" will of course depend on your specific goals, but here are a few relatively "standard" methods:

  • Equal error rate (EER) point: the point where precision equals recall. This feels to some people like a "natural" operating point.

  • A refined and more principled version of the above is to specify cost of the different kind of errors and optimize that cost. Say misclassifying an item (an error in precision) is twice as expensive as missing an item completely (error in recall). Then the best operating point is that where (1 - recall) = 2*(1 - precision).

  • In some problems people have a natural minimal acceptable rate of either precision or recall. Say you know that if more than 20% of retrieved data is incorrect, the users will stop using your application. Then it is natural to set precision to 80% (or a bit lower) and accept whatever recall you have at that point.

corrected a typo related to the meaning of precision
Source Link

The definition of "optimal" will of course depend on your specific goals, but here are a few relatively "standard" methods:

  • Equal error rate (EER) point: the point where precision equals recall. This feels to some people like a "natural" operating point.

  • A refined and more principled version of the above is to specify cost of the different kind of errors and optimize that cost. Say misclassifying an item (an error in precision) is twice as expensive as missing an item completely (error in recall). Then the best operating point is that where (1 - recall) = 2*(1 - precision).

  • In some problems people have a natural minimal acceptable rate of either precision or recall. Say you know that if more than 20% of retrieved data is incorrect, the users will stop using your application. Then it is natural to set precision to 20%80% (or a bit lesshigher) and accept whatever recall you have at that point.

The definition of "optimal" will of course depend on your specific goals, but here are a few relatively "standard" methods:

  • Equal error rate (EER) point: the point where precision equals recall. This feels to some people like a "natural" operating point.

  • A refined and more principled version of the above is to specify cost of the different kind of errors and optimize that cost. Say misclassifying an item (an error in precision) is twice as expensive as missing an item completely (error in recall). Then the best operating point is that where (1 - recall) = 2*(1 - precision).

  • In some problems people have a natural minimal acceptable rate of either precision or recall. Say you know that if more than 20% of retrieved data is incorrect, the users will stop using your application. Then it is natural to set precision to 20% (or a bit less) and accept whatever recall you have at that point.

The definition of "optimal" will of course depend on your specific goals, but here are a few relatively "standard" methods:

  • Equal error rate (EER) point: the point where precision equals recall. This feels to some people like a "natural" operating point.

  • A refined and more principled version of the above is to specify cost of the different kind of errors and optimize that cost. Say misclassifying an item (an error in precision) is twice as expensive as missing an item completely (error in recall). Then the best operating point is that where (1 - recall) = 2*(1 - precision).

  • In some problems people have a natural minimal acceptable rate of either precision or recall. Say you know that if more than 20% of retrieved data is incorrect, the users will stop using your application. Then it is natural to set precision to 80% (or a bit higher) and accept whatever recall you have at that point.

Source Link

The definition of "optimal" will of course depend on your specific goals, but here are a few relatively "standard" methods:

  • Equal error rate (EER) point: the point where precision equals recall. This feels to some people like a "natural" operating point.

  • A refined and more principled version of the above is to specify cost of the different kind of errors and optimize that cost. Say misclassifying an item (an error in precision) is twice as expensive as missing an item completely (error in recall). Then the best operating point is that where (1 - recall) = 2*(1 - precision).

  • In some problems people have a natural minimal acceptable rate of either precision or recall. Say you know that if more than 20% of retrieved data is incorrect, the users will stop using your application. Then it is natural to set precision to 20% (or a bit less) and accept whatever recall you have at that point.