A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

learn more… | top users | synonyms

0
votes
0answers
11 views

Difference between minimize risk and minimize misclassification probability

I am currently reading the book "Neural Networks for Pattern recognition". In Chapter 1.10, it said So far we have based our classification decisions on the desire to minimize the probability of ...
1
vote
1answer
21 views

Selecting a loss-function for k-fold cross-validation over shrinkage parameter

I am doing a penalized regression with categorical (ordinal) outcomes. I would like to select the shrinkage parameter $\lambda$ on the basis of cross-validation (CV). In this case, I have 50k ...
1
vote
1answer
34 views

Intuitive explanation of logloss

In several kaggle competitions the scoring was based on "logloss". This relates to classification error. Here is a technical answer but I am looking for an intuitive answer. I really liked the ...
0
votes
0answers
41 views

What loss function for multi-class, multi-label classification tasks in neural networks?

I'm training a neural network to classify a set of objects into n-classes. Each object can belong to multiple classes at the same time (multi-class, multi-label). I read that for multi-class problems ...
2
votes
1answer
47 views

Probabilistic classification and loss functions

I am trying to compare several binary classifiers. These classifiers (Gaussian Processes in my case, but it shouldn't matter) give me probabilistic predictions. Let's introduce some notations: $$y_i ...
0
votes
0answers
25 views

Is it okay to use cross entropy loss function with soft labels?

I have a classification problem where pixels will be labeled with soft labels (which denote probabilities) rather than hard 0,1 labels. Earlier with hard 0,1 pixel labeling the cross entropy loss ...
0
votes
1answer
58 views

Relationship between 0-1 Loss and error Type I and II in Neyman Pearson

In the context of hypothesis test $$H_0:\theta\in \Theta_0$$ $$H_1:\theta\notin \Theta_0$$. Find the relationship between the 0-1 loss defined by $$L(\theta,\delta)= \begin{cases} 1-\delta ...
9
votes
0answers
79 views

Can cross-validation be helpful if we are interested only in modeling, not in forecasting?

Can cross-validation be helpful if we are interested only in modeling (i.e. estimating parameters), not in forecasting? I see how cross-validation is extremely useful if your goal is to make good ...
6
votes
1answer
150 views

Loss function Approximation With Taylor Expansion

As an example, take the objective function of the XGBOOST model on the $t$'th iteration: $$\mathcal{L}^{(t)}=\sum_{i=1}^n\ell(y_i,\hat{y}_i^{(t-1)}+f_t(\mathbf{x}_i))+\Omega(f_t)$$ where $\ell$ is ...
1
vote
0answers
29 views

How to define loss function for an unbalanced dataset?

I use neural network to do classification. But instead of outputing one label, I want to ouput four independent labels such as [-1,1,1,-1]. Each of them is either 1 or -1, indicating a classification ...
8
votes
2answers
184 views

Comparison between Bayes estimators

Consider the quadratic loss $L(\theta,\delta)=(\theta-\delta)^2$, with prior given $\pi(\theta)$ where $\pi(\theta)\sim U(0,1/2)$. Let $f(x|\theta)=\theta x^{\theta-1}\mathbb{I}_{[0,1]}(x), ...
1
vote
1answer
76 views

Is it common practice to minimize the mean loss over the batches instead of the sum?

Tensorflow has an example tutorial about classifying CIFAR-10. On the tutorial the average cross entropy loss across the batch is minimized. ...
3
votes
1answer
87 views

Training loss goes down and up again. What is happening?

My training loss goes down and then up again. It is very weird. The cross-validation loss tracks the training loss. What is going on? I have two stacked LSTMS as follows (on Keras): ...
3
votes
1answer
43 views

Error distributions and loss functions

If I'm right, using the Gaussian distribution in the maximum likelihood estimate yields the mean squared loss. Are there similar relationships between other distributions and losses (say Bernoulli ...
2
votes
0answers
13 views

Interplay of Training Class Sizes, Class Weights, Loss function and Decision Threshold

I am facing a two-class classification problem where: There is way more training data in class 1 than in class 0. Classifying a class 0 event as class 1 has a higher loss than classifying a class 1 ...
0
votes
0answers
19 views

Loss function selection for weighting errors differently

I am building a regression model where I want to score/optimize/train 'over-predictions' to be twice costly as under predictions. I am attempting to do this in R and hopefully with caret package. ...
0
votes
0answers
28 views

Intuition for Normalized Squared Loss error function?

In terms of optimization squared loss is perhaps the most common error function used for regression. I've seen another function named "Normalized Squared Loss" mentioned, described as The ...
1
vote
1answer
32 views

Weights and hinge loss/linear SVM

Say I have a dataset with this distribution Class A: 10 Examples Class B: 100 Examples Class C: 1000 Examples Hence, i am trying to build a classifier using linear SVM. Baring all concerns about ...
0
votes
0answers
54 views

Loss function for multiple predictions without ground truth

I'm searching for a loss function to determine the difference between two predictions made by several multi-label classifiers. Consider the following: Prediction of classifier A is [0.2,0.5,0.1,0.8] ...
0
votes
0answers
43 views

Loss Function for Multinomial Logistic Regression - Cannot find its derivative

For Multinomial Logistic Regression we can define the Loss Function in the following way: $J(\theta)=\frac{-1}{m}\sum\limits_{i=1}^m\sum\limits_{j=1}^k ...
0
votes
1answer
40 views

Predicting values from linear regression

I've got a data set consisting of olympic years and the winning times for the womens 100m. I can plot a line throught the data using matlab as such: ...
1
vote
1answer
21 views

How do I show that the mean of the posterior density minimizes this squared error loss function?

This exercise comes from Koop's Bayesian Econometrics. Given $\theta$, the parameter(s) of a model (in this case $\theta$ is a scalar), $\tilde{\theta}$, the point estimate of $\theta$, and constants ...
0
votes
0answers
15 views

Deducing the regression function using the Squared Error Loss Function [duplicate]

I am reading Elements of Statistical Learning, and came across a deduction which I cannot understand. In the second chapter, the author defines the squared error loss and deduces the conditional ...
3
votes
0answers
45 views

Bias-variance decomposition

In section 3.2 of Bishop's Pattern Recognition and Machine Learning, he discusses the bias-variance decomposition, stating that for a squared loss function, the expected loss can be decomposed into a ...
0
votes
0answers
22 views

Perceptron Inseparable Case - Hinge loss

I am trying to understand the case where the perceptron algorithm can't find a perfect seperator. I am trying to understand how we bound the number of mistakes by using the hinge loss. My intuition ...
0
votes
1answer
12 views

How to target data gathering to minimize loss-function?

I have a data-set, a model (single variable) and a loss function. I can collect more data but each data point requires significant analysis to obtain. Hence how can I target the data collection to ...
0
votes
0answers
79 views

Deriving the gradient of a loss function for generalized logistic regression

I am trying, without much success so far, to derive the gradient of the following cost function in order to fit a logistic curve to some data: $J(a, k, b, m) = \sum_i^n(y_i - a + \frac{k - a}{(1 + ...
0
votes
0answers
47 views

What loss function should one use to get a high precision or high recall binary classifier?

I'm trying to make a detector of objects that occur very rarely (in images), planning to use a CNN binary classifier applied in a sliding/resized window. I've constructed balanced 1:1 ...
1
vote
1answer
24 views

Looking for an error measure like MAE that weights overprediction more than underprediction

I am trying to evaluate my prediction results using common error measures like the MAE, MSE or RMSE. For me it is much worse if the predicted value is higher than the true value. If it is less, it is ...
3
votes
0answers
34 views

Selection of Loss Function for Weather Insurance

I am using machine learning to predict agricultural yields using weather variables as inputs. I have been thinking about what loss function to optimize. I have been using RMSE thus far. The ...
0
votes
1answer
45 views

How can logistic loss return 1 for x = 0?

I have looked at the logistic loss function at many different sources, and many places I find it plotted like shown here: Taken from ...
3
votes
2answers
207 views

What loss function should I use for binary detection in face/non-face detection in CNN?

I want to use deep learning to train a face/non-face binary detection, what loss should I use, I think it is SigmoidCrossEntropyLoss or Hinge-lossenter link description here is that right, but I also ...
0
votes
0answers
12 views

Assessing a vector of errors in modeling

The quality of a model is often assessed based on a figure of merit such as RMSE. This reduces the individual errors in the model to a single number without assessing the errors as a population of ...
0
votes
0answers
95 views

Siamese networks and backpropagation

I have a misunderstanding how backpropagation works in siamese networks. Here is similar topic but it doesn't cover my question. How does the back-propagation work in a siamese neural network? ...
0
votes
0answers
39 views

Hinge loss in siamese network

I'm trying to implement method of comparing images' patches using siamese neural networks described here http://arxiv.org/pdf/1510.05970v1.pdf Authors propose using siamese CNN and they define hinge ...
7
votes
1answer
97 views

Constructing a problem-specific loss function

Problem Description I'm beginning network construction for a problem that I feel could have a far more insightful loss function than a simple MSE regression. My problem deals with multi-category ...
0
votes
0answers
48 views

Confusion in Training Logistic Regression Model

I have a confusion in Cost function in Logistic Regression, I have a set of data ...
1
vote
1answer
41 views

Determine the best sample size for minimum expected loss

Let $\theta \sim Gamma(1,2)$ and $X_1,...,X_n$ iid such that $X_i|\theta \sim Poisson(\theta)$. It is asked to determine the best sample size $n^*$ such that the posteriori risk $$L(\theta, d) = ...
3
votes
1answer
45 views

Quantreg : Unbalanced residuals

I'm trying to use the quantreg package to fit an exponential curve. Here is a reproductible example. IRL I have much more complex data with outliers, that's why I ...
3
votes
4answers
89 views

Logistic regression loss with continuous labels

I want to learn a logistic regression model but my outcome variable can take continuous values in [0;1], not only binary labels {0;1}. My model thus still needs a logistic link (to bound my ...
3
votes
0answers
95 views

CNN - Extract visual information via Gradient Descent with Backpropagation

I'm trying to reproduce the results from this paper: Mahendran, Aravindh, and Andrea Vedaldi. "Understanding deep image representations by inverting them." arXiv preprint arXiv:1412.0035 (2014). One ...
3
votes
1answer
158 views

Quadratic loss function implying conditional expectation

I am reading Bishop's pattern recognition book. In the decision theory part he first derives that using a quadratic loss function implies that our estimate $y(x)$ should be the conditional expectation ...
0
votes
0answers
56 views

Expected loss function in matlab

I have two matrices: The true one and the approximated one. In fact I want to compute the expected loss of the difference, that is, $E[||\Sigma_{approximated} - \Sigma_{true}||]$. In matlab, I ...
0
votes
0answers
50 views

Hinge loss values SVM interpretation

I'm using WEKA SPegasos SVM algorithm with Hinge Loss and I'm getting such Hinge Loss values. How to interpret those values ?? Are the values with the biggest absolute value the most significiant ?? ...
0
votes
0answers
47 views

Bayesian Decision Theory: Expected Loss

This question involves Bayesian decision theory and I have to try and derive the expected loss function. I'm not sure how to even approach this, so any help on how to begin this would be very much ...
2
votes
1answer
21 views

Struggling with payoff matrix

I've been struggling finding the loss functions, $L(\theta,d_1)$ and $L(\theta,d_2)$, for the following question: Items I manufacture are either independently flawed with probability, $p$, or ...
1
vote
0answers
38 views

Loss Size Index Function of A Lognormal Random Variable

I have this tutorial question and I've gone through the solutions, getting all but one line of working. I broke down the question to this point but I can't seem to get out the following. So Loss Size ...
0
votes
0answers
51 views

Changing optimization algorithm while optimizing

I am currently training some convolutional neural networks with cross-entropy loss. Thus, the function I am optimizing is non-convex, and at the moment I am using an optimization algorithm called Adam ...
1
vote
1answer
99 views

How can I force my model to predict the samples that are close to zero?

I have a large amount of inventory data and I am trying to predict when the inventory gets low using one component of the change in inventory (yes I know this doesn't describe inventory very well by ...
0
votes
0answers
61 views

derive loss function for gamma regression

In the R package mboost there is a family called "GammaReg" which implementes "negative Gamma log-likelihood with logarithmic link function". Still, I don't really ...