Linked Questions

1 vote
1 answer
492 views

How would you find a p threshold for a binary classification prediction? [duplicate]

Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\} $ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false ...
user avatar
  • 11
204 votes
10 answers
105k views

Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
user avatar
  • 117k
75 votes
4 answers
51k views

Reduce Classification Probability Threshold

I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will ...
user avatar
24 votes
1 answer
3k views

What is the importance of probabilistic machine learning?

I am attending a course on "Introduction to Machine Learning" where a large portion of this course to my surprise has a probabilistic approach to machine learning (ML), like a probabilistic ...
user avatar
  • 421
14 votes
3 answers
2k views

How would I bias my binary classifier to prefer false positive errors over false negatives?

I've put together a binary classifier using Keras' Sequential model. Of its errors, it predicts with false negatives more frequently than false positives. This tool is for medical application, where I'...
user avatar
30 votes
2 answers
5k views

Choosing among proper scoring rules

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
user avatar
  • 5,478
17 votes
3 answers
2k views

Brier Score and extreme class imbalance

Since I've heard about proper scoring rules for binary classification like the Brier score or Log Loss, I am more and more convinced that they are drastically underrepresented in practice in favor of ...
user avatar
  • 173
8 votes
2 answers
8k views

Best way to reduce false positive of binary classification to exactly 0?

I'm working on a task that even a 0.00001 fp rate is not acceptable, because detecting something as a positive when its not will have very bad consequences in this task, so it needs to be exactly 0 ...
user avatar
16 votes
2 answers
897 views

Why not use evaluation metrics as the loss function?

Most algorithms use their own loss function for optimization. But these loss functions are always different from metrics used for actual evaluation. For example, for building binary classification ...
user avatar
  • 835
5 votes
3 answers
580 views

Interpretation of coefficients in a poorly performing GLM

Suppose that I have trained a logistic regression model on a certain dataset, and I wish to interpret the coefficients of this model. Does it make any difference on the validity of the interpretation ...
user avatar
  • 61
14 votes
1 answer
1k views

Why is LogLoss preferred over other proper scoring rules?

It seems anytime people care about estimating probabilities accurately they choose LogLoss as the evaluation metric. But there are many other evaluation metrics which will prefer accurate estimation ...
user avatar
  • 320
13 votes
3 answers
435 views

How do you know that your classifier is suffering from class imbalance?

In cases where there is a substantial difference in relative class frequencies, it could be that the density of the minority class is never higher than the density of the majority class anywhere in ...
user avatar
4 votes
2 answers
331 views

Prediction using a logistic regression model

Given a logistic regression model: $y \in \{0, 1\}$ $ P(y=1|x;\theta) = h_{\theta}(x) = \frac{1}{1+\exp(-\theta^T x)}$ And given the value $\theta^*$ which maximises the conditional likelihood $P(y|X; ...
user avatar
  • 151
0 votes
2 answers
2k views

What is the loss function for regression using artificial neural network?

For classification, one of the most common loss functions for artificial neural networks (ANN) is cross-entropy. What about in ANN for regression? and why is cross-entropy hardly even discussed in ANN'...
user avatar
  • 3,139
5 votes
2 answers
313 views

Which class to define as positive in unbalanced classification

In a classification task, why do we usually choose the minority class as "positive" case for response variable? For example, if there are 1000, 9000 for class A and B respectively, we ...
user avatar

15 30 50 per page
1
2 3 4 5
7