1
$\begingroup$

we have system that rank the best 3 products from inventory of 20 products. Order of 3 product selected doesn't matter. one way to validate is based on recall and precision.

But each product has different price points. I addition to recall we need to understand the impact of revenue. How to perform this validation(we have training and validation data set)?

$\endgroup$
1
$\begingroup$

You can use the idea of a cost matrix. In many cases we give the same error to false positive and a false negative. However, sometimes we have different goals.

In your case, assigning to a false negative the profit from the product sounds reasonable. The cost of a false positive might be based on the reduced probability that a customer with many options will buy any of them.

Suppose that you compute the costs of the errors (usually, estimations are good enough) and you decided that for your use case a false negative costs 10 times more than a false positive.

You do the validation as usual, (e.g., doing cross validation and getting a confusion matrix). Now you should just weight by the costs when measuring the samples.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.