Questions tagged [log-loss]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2
votes
2answers
46 views

Understanding cross entropy loss

The formula for cross entropy loss is this: $$-\sum_iy_i \ln\left(\hat{y}_i\right).$$ My question is, what is the minimum and maximum value for cross entropy loss, given that there is a negative sign ...
2
votes
2answers
36 views

Classification Models giving probabilities at extreme end only

I am building a binary classification model with proportion of 1 is at only 3% and total 70000 data points.I have 5 variables out of which 3 are coming out to be important. I have built model using ...
0
votes
0answers
7 views

Output value of a gradient boosting decision tree node that has just a single example in it

The general gradient boosting algorithm for tree-based classifiers is as follows: Input: training set ${\displaystyle \{(x_{i},y_{i})\}_{i=1}^{n},}{\displaystyle \{(x_{i},y_{i})\}_{i=1}^{n},}$a ...
1
vote
1answer
46 views

Logarithmic loss vs Brier score vs AUC score

I have a dataset with two classes of elements. I also have two methods which assign (complementary) probabilities to each element in the dataset of belonging to either class. Given that I work with ...
0
votes
1answer
37 views

Negative Log-Likelihood Loss with Gibbs distribution for beta approaching infinity

This might be more of a math question, but since it concerns ML, I'll ask it here. In "A tutorial on energy-based learning" (LeCun et al., 2006), on page 15, section 2.2.4 about the Negative Log-...
1
vote
2answers
32 views

Use of sigmoidal function in output layer to predict continuous values and correct use of the loss function

Lets supose that a neural network is utilized to map a set of training data to a continuous interval between 0 and 1 utilizing a sigmoidal function on its output layer. Is it correct to optimize the ...
1
vote
0answers
24 views

binary cross entropy vs multi cross entropy

i am new to neural networks I know that multi class entropy is same as binary class entropy when the categories are only (0,1), but can some one explain it mathematically with an example that ...
0
votes
0answers
19 views

Gaussian MLP output maximum Likelihood optimization

I have problems traing networks using a Gaussian MLP as output. The output is split into on fully-connected layer that estimates expectation and another that estimates log(variance). See Auto encoding ...
0
votes
0answers
25 views

Loss computation for Gaussian MLP output layer of Variational Auto Encoder

I use a Variational Auto Encoder with Gaussian MLP as output according to Auto encoding variational Bayes appendix C I wanted to use the bound: I calculate the reconstruction loss by first ...
0
votes
1answer
31 views

How to compare log-loss across similar classification models with different baseline probabilities?

Suppose I have two datasets, A and B, which share a feature vector $X$ but have different units of analysis (e.g. people from two different countries). I have trained classifiers with the same model ...
0
votes
0answers
183 views

Why Log loss, AUC and precision & recall change differently when class imbalance problem is tackled?

I have a dataset and I'm working on a binary classification task with it. It has a class imbalance problem where False class versus ...
0
votes
0answers
78 views

Can boosting be defined through a modification of weights in a binary classification problem?

Let's assume that we have a binary classification problem (our labels are only 0 and 1). We try to find a model that generates probabilities to observer 1. We measure quality of the model by log loss: ...
0
votes
0answers
32 views

How to interpret plot that compares log loss, hinge loss, and squared error loss

In books and articles that compare different loss functions, authors very often make the following plot. The following comes from Bishop's PRML book, with the caption Plot of the ‘hinge’ error ...
1
vote
1answer
275 views

How does Cross-Entropy (log loss) work with backpropagation?

I am having some trouble understanding how Cross Entropy would work with backpropagation. For backpropagation we exploit the chain rule to find the partial derivative of the Error function in terms of ...
2
votes
0answers
110 views

derivative of cross entropy yields log-odds, does that make sense?

I am looking for a proof how to derive the logistic regression from cross-entropy loss, i.e. derive the form of a sigmoid from cross entropy. my thoughts are these: $\ell = y_i \ln{p_i} + (1-y_i)\ln{...
0
votes
0answers
66 views

Increasing Mean Intersection Over Union with Increasing Validation Loss - Semantic Segmentation

Firstly I'm new to Cross Validated so I apologize if this is structured incorrectly or I didn't find some related post or missed out something. I'm training deep networks for semantic segmentation ...
0
votes
0answers
46 views

Evaluating a model with Log Loss

I have been looking at alternative ways to intuitively understand the "goodness" of probability predictions from 2-class logistic regression models (and other ML classification models) and came ...
0
votes
0answers
19 views

Is this expression for Loss valid?

The negative likelihood loss over training set $t$ where a training instance is given by $x^{(t)}$, taget by $y^{(t)}$, a specific masking (dropout) by $m$ and weights by $w$ as: $$L(w|m) = -\sum_t ...
1
vote
0answers
24 views

How does one go about evaluating a new loss function? [closed]

Suppose a new loss function similar to log loss is proposed. Now we have to check whether it'll work in practical scenarios in general and tractable. What experiments one should run get to some ...
5
votes
1answer
935 views

Pytorch Cross Entropy Loss implementation counterintuitive

there is something I don't understand in the PyTorch implementation of Cross Entropy Loss. As far as I understand, theoretical Cross Entropy Loss is taking log-softmax probabilities and output a real ...
7
votes
3answers
1k views

Why is binary cross entropy (or log loss) used in autoencoders for non-binary data

I am working on an autoencoder for non-binary data ranging in [0,1] and while I was exploring existing solutions I noticed that many people (e.g., the keras ...
0
votes
0answers
64 views

How is the minimum logarithmic loss calculated when initializing the XGBoost algorithm?

Suppose there are $5$ sample units, $2$ of which carry the feature $y=1$ to be predicted and three of which carry the feature $y=0$. So, $2$ are positive. The XGBoost algorithm initializes with $\...
1
vote
1answer
1k views

How to achieve variational autoencoder (VAE) with unrestricted input?

For a normal VAE an input and a reconstruction with values in the range of $[0, 1]$ are expected. This is necessary since the log loss only makes sense for this range. If the input is not within $[0, ...
1
vote
0answers
73 views

Binary cross etropy loss with non binary ground truth data [closed]

Is it possible to use binary cross etropy loss with non binary ground truth data, i.e. not [0,1] values, but [0,0.1,0.5,1.0] ...
2
votes
1answer
770 views

Intuition behind logloss function

I have a difficulty understanding the intuition behind the logloss function since it seems to totally ignore negative examples where y = 0. The images below visualize my question to some extend: ...
2
votes
1answer
1k views

calculate binomial deviance (binomial log-likelihood) in the test dataset

I'm predicting probabilities $P(Y=1)$ using a probability forest (ranger in R). I want to evaluate my predictions $\hat p_i$ in a test dataset by calculating average binomial deviance (log likelihood)....
1
vote
0answers
88 views

Describing relative performance of models on the basis on log-loss / cross-entropy

Suppose I train classification algorithms on a given dataset with three model specifications: A, B, and C. On validation data, model A has an average log-loss of 1.0, model B has a log-loss of 0.75, ...
0
votes
0answers
2k views

How to implement one vs rest classifier in a multiclass classification problem?

I have a dataset which contains 750 data points with 8 classes in the target variable. I tried implementing simple machine learning models and also did hyperparameter tuning but they results were not ...
0
votes
0answers
728 views

How to minimize this loss function?

I have a loss function $L = \sum_i I(\text{log_loss}_i > 0.6)$, where $I$ is the indicator function (equals 0 when argument is false, 1 otherwise) and log_loss$_i$ is the log loss of the $i^{th}$ ...
1
vote
0answers
285 views

Log loss when classes are -1 and +1 [duplicate]

We can calculate the log loss for a classification problem with two classes as follows: where y is the label of the actual class and ...
0
votes
0answers
146 views

Difference between using thresholds to classify and using a custom objective function

Suppose we are faced with a binary classification problem. The standard approach seems to be to fit a probability estimator using a loss function like log loss and then afterwards determine the ...
4
votes
1answer
1k views

Is there a cross-entropy-like loss function for multiple classes where misclassification costs are not identical?

For this conversation I'll use the below definition of cross-entropy where there are N samples, M different classes, $ y_{ij} $ is 1 if sample i is of class j and 0 otherwise and $p_{ij}$ is the ...
1
vote
0answers
598 views

Gradient boosting for binary outcome - terminal nodes estimate (using R gbm)

I have been searching for the answer for the below query quite a long time and found a few answers (see: interpretation of gbm single tree prediction in pretty.gbm.tree or R Package GBM - Bernoulli ...
1
vote
1answer
265 views

Interpreting Inf (Infinity) as answer to logloss function?

so I am using this logloss function logLoss = function(pred, actual){ -1*mean(log(pred[model.matrix(~ actual + 0) - pred > 0])) } sometimes it is correctly ...
16
votes
1answer
10k views

logloss vs gini/auc

I've trained two models (binary classifiers using h2o AutoML) and I want to select one to use. I have the following results: ...
5
votes
1answer
8k views

Log Loss function in scikit-learn returns different values

I have been trying to wrap my head around the log loss function for model evaluation. I understand how the value is calculated after doing the math by hand. In the python module ...
5
votes
1answer
450 views

Smoothing/shrinking the predicted probability of a classifier to reduce live logloss

Let us assume we work on a 2 -class classification problem. In my setting the sample is balanced. To be precise it is a financial markets setting where up and down have approximately 50:50 chance. The ...
1
vote
1answer
873 views

Probability Calibration messes Reliability

I have about 1000 samples with 20 features and I'm using Random Forest to predict a binary class. I'm trying to apply the probability calibration process as described on scikit using ...
1
vote
1answer
346 views

Sudden increase in log-loss on training set with H2O GBM

I am performing a hyperparameter grid search for a GBM classifier in H2O, running version 3.10.4.8 (on top of python 3.5.3). This is a multiclass problem (~40 classes). As a first test, I tried a ...
2
votes
2answers
2k views

Performance Metrics for Imbalanced Classification

I'm trying to fit multiple Stochastic Gradient Descent models to a dataset where the target (binary target, 0 or 1) is very imbalanced, i.e the success rate is about 0.0001. Out of all the models I'...
1
vote
1answer
888 views

logloss equivalent for poisson regression

I have a Poisson regression model, and I would like to measure the discrepancy between actual counts and predicted counts. For binary classification model, the log-loss metrics fits for this purpose. ...
17
votes
4answers
23k views

What's considered a good log loss?

I'm trying to better understand log loss and how it works but one thing I can't seem to find is putting the log loss number into some sort of context. If my model has a log loss of 0.5, is that good? ...
0
votes
1answer
4k views

How do I choose the right loss function , logarithmic loss for 0/1 or exponential loss for -1/1? [duplicate]

As we know , we have two kinds of presentation in binary classification , one is 0/1 and the other is -1/1 . For 0/1 case , we often use "negative logarithmic likelihood" loss function for it , also ...
-1
votes
1answer
6k views

Am I correct to get negative cross_val_score when I am using neg_log_loss in scikit-learn 0.18?

I am working on a data set to predict if someone is going to donate blood from UCI Data repository The criteria of judging the solution is log loss So I implement the cross_val_score function from ...
2
votes
2answers
5k views

How does the L2 regularization penalize the high-value weights

I am reading about regularization in machine learning model. I want to understand how mathematically the L2 term penalizes the high-value weights to avoid overfitting? Any explanation?
0
votes
1answer
86 views

Log loss: model vs benchmark. Minimum/optimal value

So, I've built a multinomial logit model (with 4 classes) that has log loss equal 0.945. My benchmark model (probabilities equal classes distribution in train sample) gives log loss 1.131. How can I ...
15
votes
2answers
8k views

optimizing auc vs logloss in binary classification problems

I am performing a binary classification task where the outcome probability is fair low (aroung 3%). I am trying to decide whether to optimize by AUC or log-loss. As much as I have understood, AUC ...
1
vote
1answer
2k views

Scaling up random guess benchmark of log loss

On Quora it's said that for a random guess log loss is equal to ln(0.5) = ~0.693 That indeed implies that with a log loss of 0.69, you aren't doing any better than a random guess. OK, great. But ...
1
vote
0answers
1k views

R algorithm or function to train using a LogLoss error

Is there a way to train a RandomForest, GBM or other classification model using a LogLoss error measure? So far I have trained it, but I get an OOB error rate. From there I calculate the sigmoid and ...