Skip to main content

Questions tagged [log-loss]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
1 answer
32 views

SAS log likelihood fit statistic from SCORE statement in PROC LOGISTIC [closed]

When using a SCORE statement in PROC LOGISTIC in SAS, I can get fit statistics with FITSTAT. My response variable is binary. I want to get log likelihood, but looking at this documentation, I'm ...
cpahanson's user avatar
4 votes
1 answer
243 views

Selecting the model by bootstrapping: AIC vs. log-loss?

I'm building a predictive model with potentially multiple predictors. To that end, I try different, nested models, each with one more predictor than the previous one and compare their AICs. The AIC ...
Igor F.'s user avatar
  • 9,273
1 vote
0 answers
20 views

Log base in Cross Entropy Loss [duplicate]

What is the base for the logarithm used in the cross entropy loss (while doing multiclass classification's backpropagation)? Is it e, 2, or 10?
Sachin's user avatar
  • 111
2 votes
0 answers
52 views

Probabilistic interpretation for log-loss

Suppose I am modelling a binary response variable $Y \sim B(p)$ as a function of $p$ features $X_1, \dots, X_p$ by means of an equation of the form $$ p(Y = 1 \, | \, X = x) = f(x,\theta), $$ where $\...
Othman El Hammouchi's user avatar
18 votes
2 answers
3k views

Why not use evaluation metrics as the loss function?

Most algorithms use their own loss function for optimization. But these loss functions are always different from metrics used for actual evaluation. For example, for building binary classification ...
etang's user avatar
  • 977
3 votes
3 answers
193 views

Challenge an ICML Paper: For a given set of probability predictions and a log loss value, is the set of true labels giving such a loss unique?

Aggarwal's 2021 ICML paper "Label Inference Attacks from Log-loss Scores", seems to argue that the answer to the question in the title is "YES". The paper claims that, given ...
Dave's user avatar
  • 63.6k
2 votes
1 answer
2k views

Predicted Probability with XGBClassifier ranging only from 0.48 to 0.51 for either class

Why does my XGBClassifier predicts probability only from 0.48 to 0.51 for either class? I'm very new to XGBoost, so any suggestions are greatly appreciated! Here's ...
yuan-ning's user avatar
  • 123
5 votes
2 answers
1k views

"Dumb" log-loss for a binary classifier

I am trying to understand how I can best compare a classifier that I have trained and tuned against a "dumb" classifier, particularly in the context of binary classification with imbalanced ...
wissam124's user avatar
1 vote
1 answer
285 views

Mean logarithmic square error

I have a fallowing problems: I'm training a neural network against some set of output values (regression problem). Those values are between -inf to inf and I can't normalize them, because they come ...
Daniel Wiczew's user avatar
3 votes
1 answer
339 views

exp(log_softmax) vs softmax as neural network activation

I have read about log_softmax being more numerical stable than softmax, since it circumvents the division. I need to use softmax, probabilities between 0 and 1, for my neural network loss function. So ...
CreeperPower storing's user avatar
1 vote
0 answers
47 views

Is half the log loss, twice as good?

Lets say I have two different models based on the same dataset. Model A has a log loss of 0.30 on this dataset. Model B has a log loss of 0.60 on this dataset. If our scoring metric is log loss, is ...
WhiskeyHammer's user avatar
1 vote
1 answer
1k views

Understanding multiclass log-loss

I'm trying to understand the multiclass log-loss as described in sk-learns documentation. The wording ' Let the true labels (Y) for a set of samples be encoded as a 1-of-K binary indicator matrix...', ...
N Blake's user avatar
  • 579
0 votes
0 answers
138 views

If you're trying to match a vector $p$ to $x$, why doesn't a divisive loss function $\frac{p}{x} + \frac{x}{p}$ work better than negative log loss? [duplicate]

Suppose you had a classification problem where you are trying to predict a class label (e.g., $[0 \: 1 \: 0]^T$) with a model. One way to do this is to use log loss: $\Large \ell_{\log} = -\sum_i[y_i\...
Sam's user avatar
  • 247
2 votes
1 answer
948 views

F1 weighted vs. Log loss in SciKit learn RandomSearchCV

I am sorry to ask another question regarding this topic but I am still puzzled about the following: When I use 'F1_weighted' as my scoring argument in a ...
JonnDough's user avatar
  • 201
0 votes
0 answers
11 views

testing the accuracy of a probability logistic regression [duplicate]

I had a dataset with binary data. I created a logistic regression model with continuous data on the x-axis and binary data that has values 0 and 1 on the y-axis. Then I plotted my model, a line that ...
atilla's user avatar
  • 35
3 votes
1 answer
517 views

Proof that log-odds minimize binomial deviance

How do you prove that minimizing the binomial deviance estimates the log-odds? i.e: $$ \ln{\left ( \frac{p(x_i)}{1-p(x_i)} \right )} = \underset{f(x_i)}{argmin} \ \mathbb{E} \left [y_i \ln \left ( \...
PaulG's user avatar
  • 1,297
13 votes
2 answers
3k views

Reference for log-loss (cross-entropy)?

I'm trying to track down the original reference for the logarithmic loss (logarithmic scoring rule, cross-entropy), usually defined as: $$L_{log}=y_{true} \log(p) + (1-y_{true}) \log(1-p)$$ For the ...
Gabriel's user avatar
  • 4,342
1 vote
1 answer
436 views

Are maximizing the log probability and assigning the ground-truth token the highest rank the same?

I have been reading this paper titled Neural Text Generation with Unlikelihood Training. It is about the maximum likelihood function used to train generative models. Anyway, it says that a major flaw ...
Zain Sarwar's user avatar
3 votes
2 answers
8k views

Understanding cross entropy loss [closed]

The formula for cross entropy loss is this: $$-\sum_iy_i \ln\left(\hat{y}_i\right).$$ My question is, what is the minimum and maximum value for cross entropy loss, given that there is a negative sign ...
Bharathi A's user avatar
2 votes
2 answers
749 views

Classification Models giving probabilities at extreme end only

I am building a binary classification model with proportion of 1 is at only 3% and total 70000 data points.I have 5 variables out of which 3 are coming out to be important. I have built model using ...
pankaj negi's user avatar
13 votes
2 answers
3k views

Logarithmic loss vs Brier score vs AUC score

I have a dataset with two classes of elements. I also have two methods which assign (complementary) probabilities to each element in the dataset of belonging to either class. Given that I work with ...
Gabriel's user avatar
  • 4,342
0 votes
1 answer
364 views

Negative Log-Likelihood Loss with Gibbs distribution for beta approaching infinity

This might be more of a math question, but since it concerns ML, I'll ask it here. In "A tutorial on energy-based learning" (LeCun et al., 2006), on page 15, section 2.2.4 about the Negative Log-...
rubencart's user avatar
1 vote
2 answers
930 views

Use of sigmoidal function in output layer to predict continuous values and correct use of the loss function

Lets supose that a neural network is utilized to map a set of training data to a continuous interval between 0 and 1 utilizing a sigmoidal function on its output layer. Is it correct to optimize the ...
Marcus's user avatar
  • 71
1 vote
0 answers
209 views

binary cross entropy vs multi cross entropy

i am new to neural networks I know that multi class entropy is same as binary class entropy when the categories are only (0,1), but can some one explain it mathematically with an example that ...
Hamza.S's user avatar
  • 111
4 votes
3 answers
660 views

How to compare log-loss across similar classification models with different baseline probabilities?

Suppose I have two datasets, A and B, which share a feature vector $X$ but have different units of analysis (e.g. people from two different countries). I have trained classifiers with the same model ...
abeboparebop's user avatar
4 votes
1 answer
2k views

Why Log loss, AUC and precision & recall change differently when class imbalance problem is tackled?

I have a dataset and I'm working on a binary classification task with it. It has a class imbalance problem where False class versus ...
CyberPlayerOne's user avatar
1 vote
1 answer
2k views

How does Cross-Entropy (log loss) work with backpropagation?

I am having some trouble understanding how Cross Entropy would work with backpropagation. For backpropagation we exploit the chain rule to find the partial derivative of the Error function in terms of ...
Szász Erik's user avatar
1 vote
0 answers
795 views

Does probability calibration always enhance the logloss?

I'm trying to calibrate some probabilities returned by different classifiers. I have plotted the auc and logloss of each one and ...
Samos's user avatar
  • 920
3 votes
0 answers
335 views

derivative of cross entropy yields log-odds, does that make sense?

I am looking for a proof how to derive the logistic regression from cross-entropy loss, i.e. derive the form of a sigmoid from cross entropy. my thoughts are these: $\ell = y_i \ln{p_i} + (1-y_i)\ln{...
pikachu's user avatar
  • 753
0 votes
0 answers
22 views

Is this expression for Loss valid?

The negative likelihood loss over training set $t$ where a training instance is given by $x^{(t)}$, taget by $y^{(t)}$, a specific masking (dropout) by $m$ and weights by $w$ as: $$L(w|m) = -\sum_t ...
DuttaA's user avatar
  • 215
1 vote
0 answers
28 views

How does one go about evaluating a new loss function? [closed]

Suppose a new loss function similar to log loss is proposed. Now we have to check whether it'll work in practical scenarios in general and tractable. What experiments one should run get to some ...
user2358715's user avatar
5 votes
1 answer
2k views

Pytorch Cross Entropy Loss implementation counterintuitive

there is something I don't understand in the PyTorch implementation of Cross Entropy Loss. As far as I understand, theoretical Cross Entropy Loss is taking log-softmax probabilities and output a real ...
Julien Rhapsodos Girard's user avatar
15 votes
4 answers
6k views

Why is binary cross entropy (or log loss) used in autoencoders for non-binary data

I am working on an autoencoder for non-binary data ranging in [0,1] and while I was exploring existing solutions I noticed that many people (e.g., the keras ...
Flek's user avatar
  • 203
1 vote
0 answers
193 views

How is the minimum logarithmic loss calculated when initializing the XGBoost algorithm?

Suppose there are $5$ sample units, $2$ of which carry the feature $y=1$ to be predicted and three of which carry the feature $y=0$. So, $2$ are positive. The XGBoost algorithm initializes with $\...
R.Deilke's user avatar
1 vote
1 answer
3k views

How to achieve variational autoencoder (VAE) with unrestricted input?

For a normal VAE an input and a reconstruction with values in the range of $[0, 1]$ are expected. This is necessary since the log loss only makes sense for this range. If the input is not within $[0, ...
Spenhouet's user avatar
  • 141
1 vote
0 answers
299 views

Binary cross etropy loss with non binary ground truth data [closed]

Is it possible to use binary cross etropy loss with non binary ground truth data, i.e. not [0,1] values, but [0,0.1,0.5,1.0] ...
mrgloom's user avatar
  • 2,207
2 votes
1 answer
930 views

Intuition behind logloss function

I have a difficulty understanding the intuition behind the logloss function since it seems to totally ignore negative examples where y = 0. The images below visualize my question to some extend: ...
user8270077's user avatar
6 votes
2 answers
5k views

Calculate binomial deviance (binomial log-likelihood) in the test dataset

I'm predicting probabilities $\mathbb{P}(Y=1)$ using a probability forest (ranger in R). I want to evaluate my predictions $\hat ...
user116514's user avatar
1 vote
0 answers
182 views

Describing relative performance of models on the basis on log-loss / cross-entropy

Suppose I train classification algorithms on a given dataset with three model specifications: A, B, and C. On validation data, model A has an average log-loss of 1.0, model B has a log-loss of 0.75, ...
abeboparebop's user avatar
0 votes
0 answers
1k views

How to minimize this loss function?

I have a loss function $L = \sum_i I(\text{log_loss}_i > 0.6)$, where $I$ is the indicator function (equals 0 when argument is false, 1 otherwise) and log_loss$_i$ is the log loss of the $i^{th}$ ...
Kurt Newman's user avatar
1 vote
0 answers
318 views

Log loss when classes are -1 and +1 [duplicate]

We can calculate the log loss for a classification problem with two classes as follows: where y is the label of the actual class and ...
WJA's user avatar
  • 537
0 votes
0 answers
217 views

Difference between using thresholds to classify and using a custom objective function

Suppose we are faced with a binary classification problem. The standard approach seems to be to fit a probability estimator using a loss function like log loss and then afterwards determine the ...
the_martian's user avatar
5 votes
1 answer
3k views

Is there a cross-entropy-like loss function for multiple classes where misclassification costs are not identical?

For this conversation I'll use the below definition of cross-entropy where there are N samples, M different classes, $ y_{ij} $ is 1 if sample i is of class j and 0 otherwise and $p_{ij}$ is the ...
the_martian's user avatar
1 vote
0 answers
916 views

Gradient boosting for binary outcome - terminal nodes estimate (using R gbm)

I have been searching for the answer for the below query quite a long time and found a few answers (see: interpretation of gbm single tree prediction in pretty.gbm.tree or R Package GBM - Bernoulli ...
MarkSt's user avatar
  • 31
2 votes
1 answer
692 views

Interpreting Inf (Infinity) as answer to logloss function?

so I am using this logloss function logLoss = function(pred, actual){ -1*mean(log(pred[model.matrix(~ actual + 0) - pred > 0])) } sometimes it is correctly ...
Vivek Kulkarni's user avatar
19 votes
1 answer
20k views

logloss vs gini/auc

I've trained two models (binary classifiers using h2o AutoML) and I want to select one to use. I have the following results: ...
Dan's user avatar
  • 1,437
5 votes
1 answer
12k views

Log Loss function in scikit-learn returns different values

I have been trying to wrap my head around the log loss function for model evaluation. I understand how the value is calculated after doing the math by hand. In the python module ...
GeneticsGuy's user avatar
5 votes
1 answer
729 views

Smoothing/shrinking the predicted probability of a classifier to reduce live logloss

Let us assume we work on a 2 -class classification problem. In my setting the sample is balanced. To be precise it is a financial markets setting where up and down have approximately 50:50 chance. The ...
Richi W's user avatar
  • 3,446
2 votes
1 answer
1k views

Probability Calibration messes Reliability

I have about 1000 samples with 20 features and I'm using Random Forest to predict a binary class. I'm trying to apply the probability calibration process as described on scikit using ...
Guy Manzur's user avatar
1 vote
1 answer
512 views

Sudden increase in log-loss on training set with H2O GBM

I am performing a hyperparameter grid search for a GBM classifier in H2O, running version 3.10.4.8 (on top of python 3.5.3). This is a multiclass problem (~40 classes). As a first test, I tried a ...
Jeffrey Grover's user avatar