A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

learn more… | top users | synonyms

1
vote
0answers
16 views

Increasing the learning rate on loss function saturation

I'm currently reading about neural networks, specifically how loss functions saturation can cause problems. During my studies, I was curious if one could remedy the problem during training of neural ...
0
votes
0answers
21 views

custom loss for mean absolute percentage error (MAPE) in xgboost [on hold]

For now I do some task about regression, and in this case I use MAPE metric. I really need to try to use xgbost with custom objective (because other custom in my case has bad scores) , but i don't ...
0
votes
1answer
132 views

Gradient for logistic loss function

I would ask a question related to this one. I found an example of writing custom loss function for xgboost here: ...
3
votes
0answers
29 views

Is it ok to use symmetric loss function when evaluation metric is asymmetric?

I completely understand that it's ok to use a loss function different from the evaluation metric. For example, accuracy isn't computationally feasible to optimize directly since it's not ...
1
vote
1answer
53 views

loss function for data with noisy labels?

I'm trying to train a neural network for classification, but the labels I have are rather noisy (around 30% of the labels are wrong). The cross-entropy loss indeed works, but I was wondering are ...
1
vote
1answer
36 views

How to calculate the information loss of PCA?

How would we calculate the information loss of reducing dimensions using PCA ? Would it be the amount of variance loss if we skip certain eigenvectors after the PCA ?
0
votes
0answers
21 views

Unequal misclassification costs for SVC?

I wonder if there is a way to specify custom cost function in sklearn/python? (atm I use sklearn SVC) My real problem has 7 different classes, but to make it more clear lets assume that I want to ...
0
votes
0answers
15 views

xgboost - multi:softmax objective

This might be a overly simple question, but I just want to confirm that the multi:softmax objective in xgboost, what it really means is that it uses a softmax function with categorical cross entropy ...
0
votes
1answer
33 views

Cross entropy-equivalent loss suitable for real-valued labels

I am building a model whose outputs are between 0-1 and the goal is to minimize a cost function over the predicted values and labels. So far everything seems to be easy but my labels are real-valued ...
1
vote
1answer
26 views

How best to summarize a predictive discrete distribution in a single number?

I have generated a predictive distribution for a future discrete observable outcome, and would like to generate a single value $p$ which we would most likely encounter when we perform the experiment ...
1
vote
1answer
15 views

Loss function/error measurement for allocation problem

I'm working on a prediction rule for an allocation problem. So, it's data like this: ...
0
votes
1answer
14 views

Scaling up random guess benchmark of log loss

On Quora it's said that for a random guess log loss is equal to ln(0.5) = ~0.693 That indeed implies that with a log loss of 0.69, you aren't doing any better than a random guess. OK, great. ...
1
vote
1answer
19 views

Cost-sensitive SVM with sklearn

Is there a direct cost-sensitive implementation of the SVM classifiers (CS-SVM) within the sklearn module? There are several ad hoc methods for the cost-sensitive SVM on "the market", but I am ...
2
votes
1answer
31 views

SVM cost function: old and new definitions

I am trying to reconcile different definitions of the soft-margin SVM cost / loss function in primal form. There is a "max()" operator that I do not understand. I learned about SVM many years ago ...
0
votes
1answer
55 views

Loss functions that act on real-valued output vectors (and NOT just on 1-hot vectors)

I am trying to modify Andrej Karpathy's char-RNN code. As far as I understand, the loss function used in his code for a LSTM is the Softmax function function (in the file model/LSTM.lua ). I ...
0
votes
0answers
8 views

incorporating the concept of 'coverage' in cross entropy loss

by coverage, I mean that I want my binary classifier (neural network) to perform EXTREMELY well on a large portion of the data (e.g. 95%), even if this means that it performs extremely poorly on the ...
5
votes
1answer
92 views

Scoring quantile regressor

Let's suppose that there is a real random variable $Y$ that is generated by some random process that depends somehow on vector $\vec x.$ I've built a model that for given $\vec x$ predicts $\tau$-...
0
votes
0answers
18 views

Loss function for tags reconstruction

I am trying to build function which reconstructs recipe ingredients by subset of them and with my loss function it predict only most popular ones. May be someone can suggest better loss function. ...
2
votes
0answers
206 views

choosing a loss function for gbm

I am using gbm to predict an imbalanced binary outcome, with the intent of obtaining a ranking by class probability estimation that produces a strong class ...
1
vote
0answers
28 views

Interpretation of the consistency property of a loss function

I am looking for an interpretation of the consistency property of a loss function used for classification (e.g., the SVM's hinge loss: $V(t)=\max(0,1-t)$). I copy from Wikipedia: Furthermore, it ...
0
votes
0answers
18 views

Difference between minimize risk and minimize misclassification probability

I am currently reading the book "Neural Networks for Pattern recognition". In Chapter 1.10, it said So far we have based our classification decisions on the desire to minimize the probability of ...
1
vote
1answer
41 views

Selecting a loss-function for k-fold cross-validation over shrinkage parameter

I am doing a penalized regression with categorical (ordinal) outcomes. I would like to select the shrinkage parameter $\lambda$ on the basis of cross-validation (CV). In this case, I have 50k ...
1
vote
1answer
41 views

Intuitive explanation of logloss

In several kaggle competitions the scoring was based on "logloss". This relates to classification error. Here is a technical answer but I am looking for an intuitive answer. I really liked the ...
0
votes
0answers
222 views

What loss function for multi-class, multi-label classification tasks in neural networks?

I'm training a neural network to classify a set of objects into n-classes. Each object can belong to multiple classes at the same time (multi-class, multi-label). I read that for multi-class problems ...
2
votes
1answer
63 views

Probabilistic classification and loss functions

I am trying to compare several binary classifiers. These classifiers (Gaussian Processes in my case, but it shouldn't matter) give me probabilistic predictions. Let's introduce some notations: $$y_i \...
1
vote
1answer
72 views

Is it okay to use cross entropy loss function with soft labels?

I have a classification problem where pixels will be labeled with soft labels (which denote probabilities) rather than hard 0,1 labels. Earlier with hard 0,1 pixel labeling the cross entropy loss ...
0
votes
1answer
64 views

Relationship between 0-1 Loss and error Type I and II in Neyman Pearson

In the context of hypothesis test $$H_0:\theta\in \Theta_0$$ $$H_1:\theta\notin \Theta_0$$. Find the relationship between the 0-1 loss defined by $$L(\theta,\delta)= \begin{cases} 1-\delta & \...
9
votes
1answer
106 views

Can cross-validation be helpful if we are interested only in modeling, not in forecasting?

Can cross-validation be helpful if we are interested only in modeling (i.e. estimating parameters), not in forecasting? I see how cross-validation is extremely useful if your goal is to make good ...
6
votes
1answer
228 views

Loss function Approximation With Taylor Expansion

As an example, take the objective function of the XGBOOST model on the $t$'th iteration: $$\mathcal{L}^{(t)}=\sum_{i=1}^n\ell(y_i,\hat{y}_i^{(t-1)}+f_t(\mathbf{x}_i))+\Omega(f_t)$$ where $\ell$ is ...
1
vote
0answers
40 views

How to define loss function for an unbalanced dataset?

I use neural network to do classification. But instead of outputing one label, I want to ouput four independent labels such as [-1,1,1,-1]. Each of them is either 1 or -1, indicating a classification ...
8
votes
2answers
193 views

Comparison between Bayes estimators

Consider the quadratic loss $L(\theta,\delta)=(\theta-\delta)^2$, with prior given $\pi(\theta)$ where $\pi(\theta)\sim U(0,1/2)$. Let $f(x|\theta)=\theta x^{\theta-1}\mathbb{I}_{[0,1]}(x), \...
2
votes
1answer
152 views

Is it common practice to minimize the mean loss over the batches instead of the sum?

Tensorflow has an example tutorial about classifying CIFAR-10. On the tutorial the average cross entropy loss across the batch is minimized. ...
3
votes
1answer
132 views

Training loss goes down and up again. What is happening?

My training loss goes down and then up again. It is very weird. The cross-validation loss tracks the training loss. What is going on? I have two stacked LSTMS as follows (on Keras): ...
3
votes
1answer
43 views

Error distributions and loss functions

If I'm right, using the Gaussian distribution in the maximum likelihood estimate yields the mean squared loss. Are there similar relationships between other distributions and losses (say Bernoulli ...
2
votes
0answers
16 views

Interplay of Training Class Sizes, Class Weights, Loss function and Decision Threshold

I am facing a two-class classification problem where: There is way more training data in class 1 than in class 0. Classifying a class 0 event as class 1 has a higher loss than classifying a class 1 ...
0
votes
0answers
21 views

Loss function selection for weighting errors differently

I am building a regression model where I want to score/optimize/train 'over-predictions' to be twice costly as under predictions. I am attempting to do this in R and hopefully with caret package. ...
0
votes
0answers
48 views

Intuition for Normalized Squared Loss error function?

In terms of optimization squared loss is perhaps the most common error function used for regression. I've seen another function named "Normalized Squared Loss" mentioned, described as The ...
1
vote
1answer
40 views

Weights and hinge loss/linear SVM

Say I have a dataset with this distribution Class A: 10 Examples Class B: 100 Examples Class C: 1000 Examples Hence, i am trying to build a classifier using linear SVM. Baring all concerns about ...
0
votes
0answers
54 views

Loss function for multiple predictions without ground truth

I'm searching for a loss function to determine the difference between two predictions made by several multi-label classifiers. Consider the following: Prediction of classifier A is [0.2,0.5,0.1,0.8] ...
0
votes
0answers
63 views

Loss Function for Multinomial Logistic Regression - Cannot find its derivative

For Multinomial Logistic Regression we can define the Loss Function in the following way: $J(\theta)=\frac{-1}{m}\sum\limits_{i=1}^m\sum\limits_{j=1}^k 1(y^{(i)}=j)\log(\frac{\exp(\theta_j^{T}x^{(i)})...
0
votes
1answer
42 views

Predicting values from linear regression

I've got a data set consisting of olympic years and the winning times for the womens 100m. I can plot a line throught the data using matlab as such: ...
1
vote
1answer
22 views

How do I show that the mean of the posterior density minimizes this squared error loss function?

This exercise comes from Koop's Bayesian Econometrics. Given $\theta$, the parameter(s) of a model (in this case $\theta$ is a scalar), $\tilde{\theta}$, the point estimate of $\theta$, and constants $...
0
votes
0answers
15 views

Deducing the regression function using the Squared Error Loss Function [duplicate]

I am reading Elements of Statistical Learning, and came across a deduction which I cannot understand. In the second chapter, the author defines the squared error loss and deduces the conditional ...
3
votes
0answers
54 views

Bias-variance decomposition

In section 3.2 of Bishop's Pattern Recognition and Machine Learning, he discusses the bias-variance decomposition, stating that for a squared loss function, the expected loss can be decomposed into a ...
0
votes
0answers
23 views

Perceptron Inseparable Case - Hinge loss

I am trying to understand the case where the perceptron algorithm can't find a perfect seperator. I am trying to understand how we bound the number of mistakes by using the hinge loss. My intuition ...
0
votes
1answer
13 views

How to target data gathering to minimize loss-function?

I have a data-set, a model (single variable) and a loss function. I can collect more data but each data point requires significant analysis to obtain. Hence how can I target the data collection to ...
0
votes
0answers
120 views

Deriving the gradient of a loss function for generalized logistic regression

I am trying, without much success so far, to derive the gradient of the following cost function in order to fit a logistic curve to some data: $J(a, k, b, m) = \sum_i^n(y_i - a + \frac{k - a}{(1 + e^{...
0
votes
0answers
66 views

What loss function should one use to get a high precision or high recall binary classifier?

I'm trying to make a detector of objects that occur very rarely (in images), planning to use a CNN binary classifier applied in a sliding/resized window. I've constructed balanced 1:1 positive-...
1
vote
1answer
26 views

Looking for an error measure like MAE that weights overprediction more than underprediction

I am trying to evaluate my prediction results using common error measures like the MAE, MSE or RMSE. For me it is much worse if the predicted value is higher than the true value. If it is less, it is ...
3
votes
0answers
37 views

Selection of Loss Function for Weather Insurance

I am using machine learning to predict agricultural yields using weather variables as inputs. I have been thinking about what loss function to optimize. I have been using RMSE thus far. The ...