Linked Questions

5 votes
1 answer

Penalising Error above a certain Threshold

I have a ML model (a NN in the specific but I don't think it's important for the purpose of my question) that is doing pretty decent at his job, which is predicting the demand of a certain substance X ...
paolopazzo's user avatar
8 votes
2 answers

Is there a Good Illustrative Example where the Hinge Loss (SVM) Gives a Higher Accuracy than the Logistic Loss

Vladimir Vapnik wrote: “When solving a problem of interest, do not solve a more general problem as an intermediate step. Try to get the answer that you really need but not a more general one.” ...
Dikran Marsupial's user avatar
2 votes
1 answer

Custom metrics for multiclass classification when class errors have different weights

I have a multiclass classification problem (eg. the target variable is made by 4 different outcomes: Product A, Product B, Product C and NO Product). Not all the errors are equal: for example, if the ...
A1010's user avatar
  • 203
2 votes
0 answers

Testing for Statistical Significance of 200 million Features [closed]

I have 200 million features and 1 label (features and label have about 1 million observations). Features are binary, and each has an unknown but different amounts of True and False. Label is also ...
Franc Weser's user avatar
28 votes
1 answer

What does it mean that AUC is a semi-proper scoring rule?

A proper scoring rule is a rule that is maximized by a 'true' model and it doesn't allow 'hedging' or gaming the system (deliberately reporting different results as is the true belief of the model to ...
rep_ho's user avatar
  • 7,181
30 votes
2 answers

What is the statistical model behind the SVM algorithm?

I have learned that, when dealing with data using model-based approach, the first step is modeling data procedure as a statistical model. Then the next step is developing efficient/fast inference/...
i_love_somebody's user avatar
226 votes
10 answers

Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
Tim's user avatar
  • 130k
5 votes
2 answers

Classification accuracy based on probability

Let's say we have a simple binary classification problem. So for a predictor X we want to predict response Y. Y is binary, so either 0 or 1. Now let's say we use two different classifiers, model1 and ...
Leander Moesinger's user avatar
2 votes
0 answers

Assessing Classification Accuracy with False Positives and False Negatives

I have been reading this forum but cannot find anything specific enough to address my problem. I have classified disease in the below image (red spots), and verified disease by GPS (Red Circles). ...
Alexander's user avatar
1 vote
0 answers

When to use predictive power versus when to use model fitting metrics?

I built a binary classifier using logistic regression. But I can't seem to rationalize this in my mind. After cross validation, the model's AUC is 0.9003. But, as a sanity check, I ran a GOF (...
makansij's user avatar
  • 2,199