2
$\begingroup$

Lets say that there's a binary classification problem where $X$$R_p$ and $Y ∈ \{0,1\} $ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false prediction of Y = 0 when the outcome is Y = 1, and vice versa for $L_{falsepos}$. How would you find a threshold value so that expected loss criterion for making a prediction is equivalent to predicting $Y= 1$ if $p ≥ threshold$ and predicting $Y = 0$ otherwise.

I thought about approaching it using Neymay Pearson Test, but would there be a simpler way to do this?

$\endgroup$
2
0
$\begingroup$

You could use your ground truth to set the threshold. Create a ROC and/or a PRC, and select a threshold based on your use case.

What ratio of TP/FP is acceptable? Is recall more important then precision, or the other way around - and by how much?

This changes by use case, and is mainly up to you.

$\endgroup$
-1
$\begingroup$

in general the threshold value for a binary classification is 0.5. So Take the threshold as 0.5 and any value of the hypothesis probability greater than equal to 0.5 is y=1 else y=0.

$\endgroup$
1
  • $\begingroup$ The threshold doesn’t have to be $0.5$. $\endgroup$
    – Dave
    Nov 4 at 3:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.