Timeline for Judging a model through the TP, TN, FP, and FN values
Current License: CC BY-SA 4.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 10 at 17:04 | comment | added | ttnphns | I removed tag "metric". | |
| Jul 10 at 10:37 | comment | added | smci | Ok when you say 'model' you mean 'binary classifier'. e.g. one that predicts the existence or not existence of a "characteristic" (e.g. "there is a dog in this image"). But to evaluate the classifier performance, you have to tell us what is the relative cost of a False Positive vs a False Negative. If this was a diagnostic for a rare but fatal condition, a low FN rate is crucial but FP is not bad. Without knowing how rare/common dogs in your images are and what significance that carries, we can't say. | |
| Jul 9 at 20:43 | comment | added | ttnphns | Please check also an overview of measures stats.stackexchange.com/q/586342/3277 | |
| Jul 9 at 20:00 | answer | added | jginestet | timeline score: 2 | |
| Jul 9 at 13:46 | history | became hot network question | |||
| Jul 9 at 10:29 | answer | added | Peter Flom | timeline score: 4 | |
| Jul 9 at 6:40 | comment | added | Stephan Kolassa | The F1 score suffers from all the same issues as accuracy etc., see the links in my answer. The AUROC is better. @KansaiRobot: there absolutely are different thresholds here, it's just that they are often swept under the rug and implicitly set to 0.5, which is usually not a good choice, see this thread. You should either think carefully about your costs of "misclassification" and set your threshold accordingly, or use a model that performs well over many thresholds - through proper scoring rules. | |
| Jul 9 at 6:35 | answer | added | Stephan Kolassa | timeline score: 7 | |
| Jul 9 at 5:53 | comment | added | KansaiRobot | Thanks. I thought about it but I read the ROC curve is the plot the true positive rate against the false positive rate for different threshold values. It is this "different threshold values" what I don't understand. I got the results of the model application through their TP, etc. There are no different threshold values, are there? (sorry I am a bit confused about this) | |
| Jul 9 at 5:50 | comment | converted from answer | MarcoM | I'd add to the metrics you are alreay using the ROC curve and the F1 score. | |
| Jul 9 at 5:43 | history | asked | KansaiRobot | CC BY-SA 4.0 |