Timeline for Optimal classifier or optimal threshold for scoring
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 27, 2020 at 1:24 | vote | accept | toing | ||
| Aug 31, 2018 at 1:55 | comment | added | toing | Understood. If I were to use that partial area, will it give me a measure of average performance if the classifier was to operate with FPR in the range of 0% and 8% instead of at 8%? | |
| Aug 31, 2018 at 1:00 | comment | added | Sycorax♦ | I'm not sure that area is really what you want. What you want is the highest TPR; so just look at the TPR at 8% FPR. | |
| Aug 31, 2018 at 0:45 | comment | added | toing | Got it - and then should I look up partial AUC between 0% and 8% FPR of various classifiers and pick the one with the largest area in that range in this specific example? | |
| Aug 30, 2018 at 15:27 | comment | added | Sycorax♦ | No. Suppose your largest acceptable FPR is $10%$, but the MOE at $10\%$ is $\pm2%$. You pick the threshold that corresponds to an upper bound of 10%, which works out to 8%. You do this because otherwise there's a rather high probability that the FPR you will see in reality will exceed your maximum acceptable FPR. | |
| Aug 30, 2018 at 15:24 | comment | added | toing | Thanks. I am following up on an old answer from you: So if I know that FPR is 10% with a margin of error +- 2%, then should I look at the partial area under curve between 8% and 12% and pick the classifier that has most AUC under that limited range of FPR? | |
| Jul 31, 2018 at 1:10 | history | edited | Sycorax♦ | CC BY-SA 4.0 |
added 144 characters in body
|
| Jul 31, 2018 at 0:35 | history | answered | Sycorax♦ | CC BY-SA 4.0 |