118k views

### Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
• 128k
4k views

### What is the statistical model behind the SVM algorithm?

I have learned that, when dealing with data using model-based approach, the first step is modeling data procedure as a statistical model. Then the next step is developing efficient/fast inference/...
• 557
2k views

### What does it mean that AUC is a semi-proper scoring rule?

A proper scoring rule is a rule that is maximized by a 'true' model and it doesn't allow 'hedging' or gaming the system (deliberately reporting different results as is the true belief of the model to ...
• 7,053
4k views

### Classification accuracy based on probability

Let's say we have a simple binary classification problem. So for a predictor X we want to predict response Y. Y is binary, so either 0 or 1. Now let's say we use two different classifiers, model1 and ...
458 views

### Is there a Good Illustrative Example where the Hinge Loss (SVM) Gives a Higher Accuracy than the Logistic Loss

Vladimir Vapnik wrote: “When solving a problem of interest, do not solve a more general problem as an intermediate step. Try to get the answer that you really need but not a more general one.” ...
• 51.1k
1 vote
497 views

### Custom metrics for multiclass classification when class errors have different weights

I have a multiclass classification problem (eg. the target variable is made by 4 different outcomes: Product A, Product B, Product C and NO Product). Not all the errors are equal: for example, if the ...
• 193
60 views

### Penalising Error above a certain Threshold

I have a ML model (a NN in the specific but I don't think it's important for the purpose of my question) that is doing pretty decent at his job, which is predicting the demand of a certain substance X ...
• 317
160 views

### Assessing Classification Accuracy with False Positives and False Negatives

I have been reading this forum but cannot find anything specific enough to address my problem. I have classified disease in the below image (red spots), and verified disease by GPS (Red Circles). ...
1 vote
71 views

### When to use predictive power versus when to use model fitting metrics?

I built a binary classifier using logistic regression. But I can't seem to rationalize this in my mind. After cross validation, the model's AUC is 0.9003. But, as a sanity check, I ran a GOF (...
• 2,089