Timeline for What loss function should one use to get a high precision or high recall binary classifier?
Current License: CC BY-SA 3.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 3, 2018 at 23:30 | comment | added | Statseeker | Consider the trivial case of being very concerned about misclassification one way, but not the other - i.e. zero loss for one of the directions. The best model for that loss would predict only the class of concern. Although it would be a horrible model, the objective is achieved. It is important to understand the objective and not put blind faith in a theoretical concept (MLE) without understanding it's purpose. As noted by TravisGerke , if the emphasis is on prediction rather than modeling, then his approach is quite useful. It's certainly better than downsampling the majority class. | |
| Sep 8, 2017 at 16:33 | comment | added | Frank Harrell | No, you will disturb the calibration of the model and will get more noisy parameter estimates with the above approach. MLE exists for some very good reasons. | |
| Sep 8, 2017 at 15:54 | comment | added | Travis Gerke | Agree, but I'm not convinced that it matters if statistical inference on the parameters in the logistic regression is not the desired goal (the OP's mention of using CNN is not ML-based either). Indeed, most/all inferential output from this weighted approach would best be ignored, but the model and resulting risk scores could still be applied to a validation set with desirable results, e.g. good discrimination/calibration. | |
| Sep 8, 2017 at 15:25 | comment | added | Frank Harrell | But that would no longer be a maximum likelihood estimator - a statistical no-no | |
| Sep 8, 2017 at 15:17 | review | Late answers | |||
| Sep 8, 2017 at 15:18 | |||||
| Sep 8, 2017 at 15:07 | review | First posts | |||
| Sep 8, 2017 at 17:05 | |||||
| Sep 8, 2017 at 15:01 | history | answered | Travis Gerke | CC BY-SA 3.0 |