Questions tagged [log-loss]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
15 views

Understanding multiclass log-loss

I'm trying to understand the multiclass log-loss as described in sk-learns documentation. The wording ' Let the true labels (Y) for a set of samples be encoded as a 1-of-K binary indicator matrix...', ...
0
votes
0answers
63 views

If you're trying to match a vector $p$ to $x$, why doesn't a divisive loss function $\frac{p}{x} + \frac{x}{p}$ work better than negative log loss? [duplicate]

Suppose you had a classification problem where you are trying to predict a class label (e.g., $[0 \: 1 \: 0]^T$) with a model. One way to do this is to use log loss: $\Large \ell_{\log} = -\sum_i[y_i\...
0
votes
0answers
99 views

Why cross-entropy log loss not penalize for label=0?

Question Why not to use negative/0 labels for cross-entropy log loss in some cases, and what are such situations? Background There are cases where only positive/1 labels are used in cross-entropy log ...
1
vote
1answer
195 views

F1 weighted vs. Log loss in SciKit learn RandomSearchCV

I am sorry to ask another question regarding this topic but I am still puzzled about the following: When I use 'F1_weighted' as my scoring argument in a ...
0
votes
0answers
7 views

testing the accuracy of a probability logistic regression [duplicate]

I had a dataset with binary data. I created a logistic regression model with continuous data on the x-axis and binary data that has values 0 and 1 on the y-axis. Then I plotted my model, a line that ...
0
votes
1answer
36 views

Proof that log-odds minimize binomial deviance

How do you prove that minimizing the binomial deviance estimates the log-odds? i.e: $$ \ln{\left ( \frac{p(x_i)}{1-p(x_i)} \right )} = \underset{f(x_i)}{argmin} \ \mathbb{E} \left [y_i \ln \left ( \...
0
votes
0answers
34 views

The effect of an imbalanced dataset on multi-class log loss in an imbalanced population

I have sampled data and labeled it as being 1 of 14 classes. This dataset is very imbalanced, e.g. I have a lot of samples for class 1 and not that many for class 14. However, this same imbalance is ...
11
votes
2answers
657 views

Reference for log-loss (cross-entropy)?

I'm trying to track down the original reference for the logarithmic loss (logarithmic scoring rule, cross-entropy), usually defined as: $$L_{log}=y_{true} \log(p) + (1-y_{true}) \log(1-p)$$ For the ...
1
vote
1answer
95 views

Are maximizing the log probability and assigning the ground-truth token the highest rank the same?

I have been reading this paper titled Neural Text Generation with Unlikelihood Training. It is about the maximum likelihood function used to train generative models. Anyway, it says that a major flaw ...
2
votes
2answers
337 views

Understanding cross entropy loss [closed]

The formula for cross entropy loss is this: $$-\sum_iy_i \ln\left(\hat{y}_i\right).$$ My question is, what is the minimum and maximum value for cross entropy loss, given that there is a negative sign ...
2
votes
2answers
109 views

Classification Models giving probabilities at extreme end only

I am building a binary classification model with proportion of 1 is at only 3% and total 70000 data points.I have 5 variables out of which 3 are coming out to be important. I have built model using ...
4
votes
2answers
487 views

Logarithmic loss vs Brier score vs AUC score

I have a dataset with two classes of elements. I also have two methods which assign (complementary) probabilities to each element in the dataset of belonging to either class. Given that I work with ...
0
votes
1answer
76 views

Negative Log-Likelihood Loss with Gibbs distribution for beta approaching infinity

This might be more of a math question, but since it concerns ML, I'll ask it here. In "A tutorial on energy-based learning" (LeCun et al., 2006), on page 15, section 2.2.4 about the Negative Log-...
1
vote
2answers
231 views

Use of sigmoidal function in output layer to predict continuous values and correct use of the loss function

Lets supose that a neural network is utilized to map a set of training data to a continuous interval between 0 and 1 utilizing a sigmoidal function on its output layer. Is it correct to optimize the ...
1
vote
0answers
59 views

binary cross entropy vs multi cross entropy

i am new to neural networks I know that multi class entropy is same as binary class entropy when the categories are only (0,1), but can some one explain it mathematically with an example that ...
1
vote
1answer
207 views

How to compare log-loss across similar classification models with different baseline probabilities?

Suppose I have two datasets, A and B, which share a feature vector $X$ but have different units of analysis (e.g. people from two different countries). I have trained classifiers with the same model ...
1
vote
0answers
548 views

Why Log loss, AUC and precision & recall change differently when class imbalance problem is tackled?

I have a dataset and I'm working on a binary classification task with it. It has a class imbalance problem where False class versus ...
1
vote
1answer
797 views

How does Cross-Entropy (log loss) work with backpropagation?

I am having some trouble understanding how Cross Entropy would work with backpropagation. For backpropagation we exploit the chain rule to find the partial derivative of the Error function in terms of ...
1
vote
0answers
367 views

Does probability calibration always enhance the logloss?

I'm trying to calibrate some probabilities returned by different classifiers. I have plotted the auc and logloss of each one and ...
3
votes
0answers
202 views

derivative of cross entropy yields log-odds, does that make sense?

I am looking for a proof how to derive the logistic regression from cross-entropy loss, i.e. derive the form of a sigmoid from cross entropy. my thoughts are these: $\ell = y_i \ln{p_i} + (1-y_i)\ln{...
0
votes
0answers
19 views

Is this expression for Loss valid?

The negative likelihood loss over training set $t$ where a training instance is given by $x^{(t)}$, taget by $y^{(t)}$, a specific masking (dropout) by $m$ and weights by $w$ as: $$L(w|m) = -\sum_t ...
1
vote
0answers
26 views

How does one go about evaluating a new loss function? [closed]

Suppose a new loss function similar to log loss is proposed. Now we have to check whether it'll work in practical scenarios in general and tractable. What experiments one should run get to some ...
5
votes
1answer
1k views

Pytorch Cross Entropy Loss implementation counterintuitive

there is something I don't understand in the PyTorch implementation of Cross Entropy Loss. As far as I understand, theoretical Cross Entropy Loss is taking log-softmax probabilities and output a real ...
10
votes
3answers
3k views

Why is binary cross entropy (or log loss) used in autoencoders for non-binary data

I am working on an autoencoder for non-binary data ranging in [0,1] and while I was exploring existing solutions I noticed that many people (e.g., the keras ...
0
votes
0answers
92 views

How is the minimum logarithmic loss calculated when initializing the XGBoost algorithm?

Suppose there are $5$ sample units, $2$ of which carry the feature $y=1$ to be predicted and three of which carry the feature $y=0$. So, $2$ are positive. The XGBoost algorithm initializes with $\...
1
vote
1answer
2k views

How to achieve variational autoencoder (VAE) with unrestricted input?

For a normal VAE an input and a reconstruction with values in the range of $[0, 1]$ are expected. This is necessary since the log loss only makes sense for this range. If the input is not within $[0, ...
1
vote
0answers
152 views

Binary cross etropy loss with non binary ground truth data [closed]

Is it possible to use binary cross etropy loss with non binary ground truth data, i.e. not [0,1] values, but [0,0.1,0.5,1.0] ...
2
votes
1answer
815 views

Intuition behind logloss function

I have a difficulty understanding the intuition behind the logloss function since it seems to totally ignore negative examples where y = 0. The images below visualize my question to some extend: ...
4
votes
1answer
2k views

calculate binomial deviance (binomial log-likelihood) in the test dataset

I'm predicting probabilities $P(Y=1)$ using a probability forest (ranger in R). I want to evaluate my predictions $\hat p_i$ in a test dataset by calculating average binomial deviance (log likelihood)....
1
vote
0answers
122 views

Describing relative performance of models on the basis on log-loss / cross-entropy

Suppose I train classification algorithms on a given dataset with three model specifications: A, B, and C. On validation data, model A has an average log-loss of 1.0, model B has a log-loss of 0.75, ...
0
votes
0answers
893 views

How to minimize this loss function?

I have a loss function $L = \sum_i I(\text{log_loss}_i > 0.6)$, where $I$ is the indicator function (equals 0 when argument is false, 1 otherwise) and log_loss$_i$ is the log loss of the $i^{th}$ ...
1
vote
0answers
293 views

Log loss when classes are -1 and +1 [duplicate]

We can calculate the log loss for a classification problem with two classes as follows: where y is the label of the actual class and ...
0
votes
0answers
168 views

Difference between using thresholds to classify and using a custom objective function

Suppose we are faced with a binary classification problem. The standard approach seems to be to fit a probability estimator using a loss function like log loss and then afterwards determine the ...
4
votes
1answer
1k views

Is there a cross-entropy-like loss function for multiple classes where misclassification costs are not identical?

For this conversation I'll use the below definition of cross-entropy where there are N samples, M different classes, $ y_{ij} $ is 1 if sample i is of class j and 0 otherwise and $p_{ij}$ is the ...
1
vote
0answers
741 views

Gradient boosting for binary outcome - terminal nodes estimate (using R gbm)

I have been searching for the answer for the below query quite a long time and found a few answers (see: interpretation of gbm single tree prediction in pretty.gbm.tree or R Package GBM - Bernoulli ...
1
vote
1answer
359 views

Interpreting Inf (Infinity) as answer to logloss function?

so I am using this logloss function logLoss = function(pred, actual){ -1*mean(log(pred[model.matrix(~ actual + 0) - pred > 0])) } sometimes it is correctly ...
16
votes
1answer
14k views

logloss vs gini/auc

I've trained two models (binary classifiers using h2o AutoML) and I want to select one to use. I have the following results: ...
5
votes
1answer
10k views

Log Loss function in scikit-learn returns different values

I have been trying to wrap my head around the log loss function for model evaluation. I understand how the value is calculated after doing the math by hand. In the python module ...
5
votes
1answer
518 views

Smoothing/shrinking the predicted probability of a classifier to reduce live logloss

Let us assume we work on a 2 -class classification problem. In my setting the sample is balanced. To be precise it is a financial markets setting where up and down have approximately 50:50 chance. The ...
1
vote
1answer
1k views

Probability Calibration messes Reliability

I have about 1000 samples with 20 features and I'm using Random Forest to predict a binary class. I'm trying to apply the probability calibration process as described on scikit using ...
1
vote
1answer
409 views

Sudden increase in log-loss on training set with H2O GBM

I am performing a hyperparameter grid search for a GBM classifier in H2O, running version 3.10.4.8 (on top of python 3.5.3). This is a multiclass problem (~40 classes). As a first test, I tried a ...
2
votes
2answers
2k views

Performance Metrics for Imbalanced Classification

I'm trying to fit multiple Stochastic Gradient Descent models to a dataset where the target (binary target, 0 or 1) is very imbalanced, i.e the success rate is about 0.0001. Out of all the models I'...
1
vote
1answer
1k views

logloss equivalent for poisson regression

I have a Poisson regression model, and I would like to measure the discrepancy between actual counts and predicted counts. For binary classification model, the log-loss metrics fits for this purpose. ...
26
votes
4answers
37k views

What's considered a good log loss?

I'm trying to better understand log loss and how it works but one thing I can't seem to find is putting the log loss number into some sort of context. If my model has a log loss of 0.5, is that good? ...
0
votes
1answer
4k views

How do I choose the right loss function , logarithmic loss for 0/1 or exponential loss for -1/1? [duplicate]

As we know , we have two kinds of presentation in binary classification , one is 0/1 and the other is -1/1 . For 0/1 case , we often use "negative logarithmic likelihood" loss function for it , also ...
-1
votes
1answer
8k views

Am I correct to get negative cross_val_score when I am using neg_log_loss in scikit-learn 0.18?

I am working on a data set to predict if someone is going to donate blood from UCI Data repository The criteria of judging the solution is log loss So I implement the cross_val_score function from ...
4
votes
2answers
6k views

How does the L2 regularization penalize the high-value weights

I am reading about regularization in machine learning model. I want to understand how mathematically the L2 term penalizes the high-value weights to avoid overfitting? Any explanation?
0
votes
1answer
92 views

Log loss: model vs benchmark. Minimum/optimal value

So, I've built a multinomial logit model (with 4 classes) that has log loss equal 0.945. My benchmark model (probabilities equal classes distribution in train sample) gives log loss 1.131. How can I ...
17
votes
2answers
11k views

optimizing auc vs logloss in binary classification problems

I am performing a binary classification task where the outcome probability is fair low (aroung 3%). I am trying to decide whether to optimize by AUC or log-loss. As much as I have understood, AUC ...
1
vote
1answer
2k views

Scaling up random guess benchmark of log loss

On Quora it's said that for a random guess log loss is equal to ln(0.5) = ~0.693 That indeed implies that with a log loss of 0.69, you aren't doing any better than a random guess. OK, great. But ...