1 vote
492 views

### How would you find a p threshold for a binary classification prediction? [duplicate]

Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\}$ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false ...
105k views

### Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
51k views

### Reduce Classification Probability Threshold

I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will ...
3k views

### What is the importance of probabilistic machine learning?

I am attending a course on "Introduction to Machine Learning" where a large portion of this course to my surprise has a probabilistic approach to machine learning (ML), like a probabilistic ...
2k views

### How would I bias my binary classifier to prefer false positive errors over false negatives?

I've put together a binary classifier using Keras' Sequential model. Of its errors, it predicts with false negatives more frequently than false positives. This tool is for medical application, where I'...
5k views

### Choosing among proper scoring rules

Most resources on proper scoring rules mention a number of different scoring rules like log-loss, Brier score or spherical scoring. However, they often don't give much guidance on the differences ...
2k views

### Brier Score and extreme class imbalance

Since I've heard about proper scoring rules for binary classification like the Brier score or Log Loss, I am more and more convinced that they are drastically underrepresented in practice in favor of ...
8k views

### Best way to reduce false positive of binary classification to exactly 0?

I'm working on a task that even a 0.00001 fp rate is not acceptable, because detecting something as a positive when its not will have very bad consequences in this task, so it needs to be exactly 0 ...
897 views

### Why not use evaluation metrics as the loss function?

Most algorithms use their own loss function for optimization. But these loss functions are always different from metrics used for actual evaluation. For example, for building binary classification ...
580 views

### Interpretation of coefficients in a poorly performing GLM

Suppose that I have trained a logistic regression model on a certain dataset, and I wish to interpret the coefficients of this model. Does it make any difference on the validity of the interpretation ...
1k views

### Why is LogLoss preferred over other proper scoring rules?

It seems anytime people care about estimating probabilities accurately they choose LogLoss as the evaluation metric. But there are many other evaluation metrics which will prefer accurate estimation ...
435 views

### How do you know that your classifier is suffering from class imbalance?

In cases where there is a substantial difference in relative class frequencies, it could be that the density of the minority class is never higher than the density of the majority class anywhere in ...
331 views

### Prediction using a logistic regression model

Given a logistic regression model: $y \in \{0, 1\}$ $P(y=1|x;\theta) = h_{\theta}(x) = \frac{1}{1+\exp(-\theta^T x)}$ And given the value $\theta^*$ which maximises the conditional likelihood \$P(y|X; ...
2k views

### What is the loss function for regression using artificial neural network?

For classification, one of the most common loss functions for artificial neural networks (ANN) is cross-entropy. What about in ANN for regression? and why is cross-entropy hardly even discussed in ANN'...