Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [log-loss]

The tag has no usage guidance.

1
vote
0answers
16 views

How does one go about evaluating a new loss function? [closed]

Suppose a new loss function similar to log loss is proposed. Now we have to check whether it'll work in practical scenarios in general and tractable. What experiments one should run get to some ...
0
votes
0answers
58 views

Balanced LogLoss with XGBoost

Following the discussion on here I started worrying less about class imbalance. However, I recently started building a predictor, using XGBoost, and I wanted to used LogLoss as my target metric. I ...
3
votes
1answer
108 views

Pytorch Cross Entropy Loss implementation counterintuitive

there is something I don't understand in the PyTorch implementation of Cross Entropy Loss. As far as I understand, theoretical Cross Entropy Loss is taking log-softmax probabilities and output a real ...
2
votes
3answers
207 views

Why is binary cross entropy (or log loss) used in autoencoders for non-binary data

I am working on an autoencoder for non-binary data ranging in [0,1] and while I was exploring existing solutions I noticed that in many people (e.g., the keras ...
0
votes
0answers
40 views

How is the minimum logarithmic loss calculated when initializing the XGBoost algorithm?

Suppose there are $5$ sample units, $2$ of which carry the feature $y=1$ to be predicted and three of which carry the feature $y=0$. So, $2$ are positive. The XGBoost algorithm initializes with $\...
0
votes
1answer
381 views

How to achieve variational autoencoder (VAE) with unrestricted input?

For a normal VAE an input and a reconstruction with values in the range of $[0, 1]$ are expected. This is necessary since the log loss only makes sense for this range. If the input is not within $[0, ...
1
vote
0answers
30 views

Binary cross etropy loss with non binary ground truth data [closed]

Is it possible to use binary cross etropy loss with non binary ground truth data, i.e. not [0,1] values, but [0,0.1,0.5,1.0] ...
2
votes
1answer
598 views

Intuition behind logloss function

I have a difficulty understanding the intuition behind the logloss function since it seems to totally ignore negative examples where y = 0. The images below visualize my question to some extend: ...
0
votes
0answers
419 views

Gradient of the cross entropy loss function

I have been puzzled by how to calculate the derivative of the following cross entropy loss function underlying my neural network: CEloss = $\frac{-1}{N} \sum_{n=1}^{N} \sum_{k=1}^{K} t_{n,k} \log y_{...
1
vote
0answers
162 views

calculate binomial deviance (binomial log-likelihood) in the test dataset

I'm predicting probabilities $P(Y=1)$ using a probability forest (ranger in R). I want to evaluate my predictions $\hat p_i$ in a test dataset by calculating average binomial deviance (log likelihood)....
1
vote
0answers
33 views

Describing relative performance of models on the basis on log-loss / cross-entropy

Suppose I train classification algorithms on a given dataset with three model specifications: A, B, and C. On validation data, model A has an average log-loss of 1.0, model B has a log-loss of 0.75, ...
0
votes
0answers
715 views

How to implement one vs rest classifier in a multiclass classification problem?

I have a dataset which contains 750 data points with 8 classes in the target variable. I tried implementing simple machine learning models and also did hyperparameter tuning but they results were not ...
0
votes
0answers
303 views

How to minimize this loss function?

I have a loss function $L = \sum_i I(\text{log_loss}_i > 0.6)$, where $I$ is the indicator function (equals 0 when argument is false, 1 otherwise) and log_loss$_i$ is the log loss of the $i^{th}$ ...
0
votes
0answers
203 views

Log loss when classes are -1 and +1

We can calculate the log loss for a classification problem with two classes as follows: where y is the label of the actual class and ...
0
votes
0answers
102 views

Difference between using thresholds to classify and using a custom objective function

Suppose we are faced with a binary classification problem. The standard approach seems to be to fit a probability estimator using a loss function like log loss and then afterwards determine the ...
4
votes
1answer
776 views

Is there a cross-entropy-like loss function for multiple classes where misclassification costs are not identical?

For this conversation I'll use the below definition of cross-entropy where there are N samples, M different classes, $ y_{ij} $ is 1 if sample i is of class j and 0 otherwise and $p_{ij}$ is the ...
1
vote
0answers
406 views

Gradient boosting for binary outcome - terminal nodes estimate (using R gbm)

I have been searching for the answer for the below query quite a long time and found a few answers (see: interpretation of gbm single tree prediction in pretty.gbm.tree or R Package GBM - Bernoulli ...
1
vote
1answer
128 views

Interpreting Inf (Infinity) as answer to logloss function?

so I am using this logloss function logLoss = function(pred, actual){ -1*mean(log(pred[model.matrix(~ actual + 0) - pred > 0])) } sometimes it is correctly ...
9
votes
1answer
6k views

logloss vs gini/auc

I've trained two models (binary classifiers using h2o AutoML) and I want to select one to use. I have the following results: ...
2
votes
1answer
4k views

Log Loss function in scikit-learn returns different values

I have been trying to wrap my head around the log loss function for model evaluation. I understand how the value is calculated after doing the math by hand. In the python module ...
4
votes
1answer
357 views

Smoothing/shrinking the predicted probability of a classifier to reduce live logloss

Let us assume we work on a 2 -class classification problem. In my setting the sample is balanced. To be precise it is a financial markets setting where up and down have approximately 50:50 chance. The ...
1
vote
1answer
593 views

Probability Calibration messes Reliability

I have about 1000 samples with 20 features and I'm using Random Forest to predict a binary class. I'm trying to apply the probability calibration process as described on scikit using ...
1
vote
1answer
256 views

Sudden increase in log-loss on training set with H2O GBM

I am performing a hyperparameter grid search for a GBM classifier in H2O, running version 3.10.4.8 (on top of python 3.5.3). This is a multiclass problem (~40 classes). As a first test, I tried a ...
2
votes
2answers
868 views

Performance Metrics for Imbalanced Classification

I'm trying to fit multiple Stochastic Gradient Descent models to a dataset where the target (binary target, 0 or 1) is very imbalanced, i.e the success rate is about 0.0001. Out of all the models I'...
0
votes
1answer
455 views

logloss equivalent for poisson regression

I have a Poisson regression model, and I would like to measure the discrepancy between actual counts and predicted counts. For binary classification model, the log-loss metrics fits for this purpose. ...
15
votes
4answers
12k views

What's considered a good log loss?

I'm trying to better understand log loss and how it works but one thing I can't seem to find is putting the log loss number into some sort of context. If my model has a log loss of 0.5, is that good? ...
0
votes
1answer
3k views

How do I choose the right loss function , logarithmic loss for 0/1 or exponential loss for -1/1? [duplicate]

As we know , we have two kinds of presentation in binary classification , one is 0/1 and the other is -1/1 . For 0/1 case , we often use "negative logarithmic likelihood" loss function for it , also ...
-2
votes
1answer
4k views

Am I correct to get negative cross_val_score when I am using neg_log_loss in scikit-learn 0.18?

I am working on a data set to predict if someone is going to donate blood from UCI Data repository The criteria of judging the solution is log loss So I implement the cross_val_score function from ...
2
votes
2answers
4k views

How does the L2 regularization penalize the high-value weights

I am reading about regularization in machine learning model. I want to understand how mathematically the L2 term penalizes the high-value weights to avoid overfitting? Any explanation?
0
votes
1answer
74 views

Log loss: model vs benchmark. Minimum/optimal value

So, I've built a multinomial logit model (with 4 classes) that has log loss equal 0.945. My benchmark model (probabilities equal classes distribution in train sample) gives log loss 1.131. How can I ...
9
votes
2answers
5k views

optimizing auc vs logloss in binary classification problems

I am performing a binary classification task where the outcome probability is fair low (aroung 3%). I am trying to decide whether to optimize by AUC or log-loss. As much as I have understood, AUC ...
1
vote
0answers
1k views

R algorithm or function to train using a LogLoss error

Is there a way to train a RandomForest, GBM or other classification model using a LogLoss error measure? So far I have trained it, but I get an OOB error rate. From there I calculate the sigmoid and ...