1
$\begingroup$

Lets say that there's a binary classification problem where $X$$R_p$ and $Y ∈ \{0,1\} $ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false prediction of Y = 0 when the outcome is Y = 1, and vice versa for $L_{falsepos}$. How would you find a threshold value so that expected loss criterion for making a prediction is equivalent to predicting $Y= 1$ if $p ≥ threshold$ and predicting $Y = 0$ otherwise.

I thought about approaching it using Neymay Pearson Test, but would there be a simpler way to do this?

$\endgroup$
2

1 Answer 1

0
$\begingroup$

You could use your ground truth to set the threshold. Create a ROC and/or a PRC, and select a threshold based on your use case.

What ratio of TP/FP is acceptable? Is recall more important then precision, or the other way around - and by how much?

This changes by use case, and is mainly up to you.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.