Questions tagged [loss-functions]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Implementing a specific GAN loss

In this paper in section 3.4 they define a GAN loss as (I simplify to the important part): $$\mathcal L_{GAN} = \max_D \min_G E_{x_{1} \sim P_1}[ \Vert D(G(x_{1})) \Vert] + \max_D E_{x_2 \sim P_2}[\...
0
votes
1answer
17 views

How to modify RMSE loss function to adopt for integer valued predictions, using a Neural Network?

Context: Prediction of dependent variables like Age, Siblings, Children, etc (which are not categorical, but bounded, and integer-valued) from a dataset using Neural Network. I'm experimenting with a ...
1
vote
0answers
6 views

Cross-entropy loss when some categories are broader than others

Let's say I want to write a classifier for pictures of dogs. Most importantly, I want to know whether something is a picture of a dog or not. Secondarily, it'd be nice to know what breed the dog is. ...
1
vote
1answer
32 views

Stability with L1 vs L2 norms

I've been looking over http://www.chioka.in/differences-between-l1-and-l2-as-loss-function-and-regularization/ and trying to get a deeper understanding of a stable vs unstable solution for L1 vs L2. ...
0
votes
0answers
7 views

Keras how to implement custom loss function that requires matrix multiplication [closed]

I am trying to implement singular value decomposition using a DNN (I'm not super concerned with whether this is even possible, whether this makes sense, etc.) This ultimately might be a silly idea, ...
0
votes
0answers
6 views

Decision making expreimeand and Prospect Theory with e greedy function in R [closed]

Hello dear statitic and r Pros, for a homework at university I have got a task that I cant handle, so I am asking for help. I dont have any experience in R and are very much in need for your help. ...
0
votes
0answers
4 views

Increasing Mean Intersection Over Union with Increasing Validation Loss - Semantic Segmentation

Firstly I'm new to Cross Validated so I apologize if this is structured incorrectly or I didn't find some related post or missed out something. I'm training deep networks for semantic segmentation ...
1
vote
1answer
44 views

Why does Perceptron use L1 norm as its error function?

Usually, people use L2 norm as a machine learning error function. per wiki the error function for a Perceptron model is ${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|}$ why is that?
0
votes
0answers
8 views

Are there some guidelines to follow while combining different types of losses to make a cost function?

I'm training an Autoencoder to reproduce the input, and the architecture is a simple fully connected neural network. The initial phase of the implementation was using float/integer dataset, and ...
0
votes
0answers
7 views

Similarity of perceptron criterion and SVM

In the book "Neural Networks and Deep Learning" by Aggarwal there is an exercise 2.10.1: Consider the following loss function for training pair $(\overline{X},y)$: $$L=max(0, a -y(\overline{W} \...
0
votes
1answer
18 views

Tensorflow 2.0/Keras - loss function with multiple inputs

I have a case where my model has multiple outputs, and I want to backpropagate the loss on one of the outputs based on a different label. Basically, the model should detect whether an object is in an ...
3
votes
1answer
24 views

Loss function to widen distribution of predictions

I'm working on a project where my business counterparts want my model's distribution of continuous predictions to more closely represent the distribution of actual values. Below is an example of what ...
0
votes
1answer
26 views

Hinge Loss: what does hinge mean?

This is terminology question, I know what is hinge loss mathematically; but I can't grok what hinge mean in hinge loss. Is it rod, or door hinge, or something else?
0
votes
0answers
18 views

How to find the optimal classifier for a given loss function?

For a binary classification problem (labels being 0 and 1) and a classifier $g$ we consider the loss function $L(g)=P[Y\neq g(X)]$. It is known that the optimal classifier $g^*$ that minimizes $L$ is $...
2
votes
1answer
57 views

how to interpret the sharp decline in loss in seq2seq models

I have a seq2seq model. I have applied this data over 20_newsgroup data set. My problem is that I face with exploding gradient ...
0
votes
1answer
22 views

What is a correct loss for a model predicting angles from images?

Backgrounds I am working with a dataset, where compass-like images are labeled with their corresponding angles from horizontal line ($0$ degrees). I am trying to make a CNN model to predict the ...
0
votes
0answers
9 views

Proving that an SVM problem with a complex loss function is convex [duplicate]

The end goal, along with proving that the problem is convex, is to be able to get the problem into a form that can be coded in CVX. I have m positively labeled data points $x_i$ $\in$ $\mathbb{R}^n, ...
1
vote
0answers
22 views

Proving that an SVM problem with a complex loss function is convex

The end goal, along with proving that the problem is convex, is to be able to get the problem into a form that can be coded in CVX. I have m positively labeled data points $x_i$ $\in$ $\mathbb{R}^n, ...
0
votes
0answers
35 views

How to use KL divergence to compare two distributions?

I am trying to model the probability distribution of a multi-dimensional dataset where all the values are discrete. Suppose the training data (represented by T) is of the shape (m, n) where n is the ...
0
votes
1answer
17 views

Validation accuracy/loss goes up and down linearly with every consecutive epoch

I'm training a CNN in keras with tensorflow backend with the following model architecture for a binary classification problem. I've divided approximately 41k images into training, validation and test ...
1
vote
0answers
21 views

Nonlinear quantile regression SSReg analogue

I have recently remembered that $SSTot = SSRes + SSReg$ fails to hold in the case of nonlinear regression. $$ y_i-\bar{y} = (y_i - \hat{y_i} + \hat{y_i} - \bar{y}) = (y_i - \hat{y_i}) + (\hat{y_i} - ...
0
votes
0answers
17 views

Solving the Binary Logistic Regression with LASSO penalty

The objective function of the Binary logistic regression with the LASSO penalty is given by, $argmin_{\beta_0,\beta}$ { $-{1}/{n}$ $\sum_{i=1}^n (Y_i({\beta_0}+{\beta^T}x_i)-log(1+exp({\beta_0}+{\...
0
votes
1answer
25 views

Why are weights being used in (generalized) dice loss, and why can't I? [closed]

Generalized dice loss is advocated as optimizing mIoU directly in semantic segmentation problems (especially those with a severe class imbalance), as opposed to other loss functions like multinomial ...
0
votes
0answers
6 views

Optimal NN architecture for regression task that benefits from classification

I am aiming to build a NN that would be optimally combining classification and regression. I have reformulated the task such that it would be less abstract and would like to know if the proposed ...
1
vote
0answers
16 views
0
votes
0answers
46 views

Autoencoder loss function - why minimise MSE?

Why are most loss functions used in autoencoder learning algorithms the mean squared error?
0
votes
0answers
16 views

How to interpret when using hinge loss performs significantly better than cross-entropy loss in a multi-class clasification problem?"

Given that hinge loss is based on the marginal loss in SVM, is there any reasonable assumption / interpretation one can make on the topology of the dataset, when using multi-class hinge loss ...
1
vote
1answer
27 views

Theory on custom loss functions for GBDT and other ML

I'm looking for resources on the theory behind choosing a loss function for ML---I'm interested in GBDT but for deep learning would work as well. I'd like to get a better understanding of how the loss ...
1
vote
0answers
36 views

Requirements of a loss function for an NN [closed]

Which requirements has to accomplish a loss function? Is differentiation the single requirement?
0
votes
0answers
7 views

creation of an objection function for an NN by means of an objective from an model which is above that NN [duplicate]

As the title above says, it is about an objective function of an NN. My issue is about a task where NN is combined with another model and has to be updated, but there are no target values. The NN is ...
2
votes
0answers
36 views

Maximizing Sum of Upper Triangle Matrix Elements with Respect to Column and Row Swapping

So, I wanna make a ranking method for teams in the EPL, there are 20 teams in EPL, therefore there are $20!$ configurations of ranking assignment, my final ranking assignment would be the one that ...
0
votes
1answer
25 views

svm loss function gradient

I was taking Stanford's cs231n class and was unable to understand the gradient calculated using the SVM loss function. You should go here to check the notes which I am talking about. This is the SVM ...
4
votes
2answers
52 views

Are loss functions necessarily additive in observations?

In all of the contexts I've seen loss functions in statistics/machine learning so far, loss functions are additive in observations. i.e.: loss $Q_D$ of dataset $D$ is an additive aggregation of losses ...
2
votes
0answers
38 views

Choosing error function for regression

I have a dataset with ~100K samples and non-negative continuous target variable. 99% of target values are zeros and the remaining 1% are right-skewed. Here are the deciles (0 and 1 correspond to min ...
1
vote
0answers
13 views

Linear regression: How to demand similar MSE across different subgroups?

In typical least square regression, we want to minimize $||y-\hat{y}||$ where $\hat{y}=B*x$ I am now working on a car fleet management problem, $y$ can be split into several groups (in my case, ...
0
votes
0answers
11 views

Loss function for one-step-ahead volatility forcasts

I'm trying to perform the MCS test using the R-package "MCS" to compare GARCH-MIDAS Models. The loss function requires as inputs a vector with some realized volatility measure ˜ σt+1 (I chose the ...
0
votes
0answers
22 views

Terminology: “L1 regularization” even if I'm using mean instead of sum? [duplicate]

In my loss function I'm using the mean of the log-cosh error between the predictions and targets, as well as an additional regularization term that scales as the mean of the absolute value of another ...
1
vote
0answers
68 views

Why using RMSE as loss function in logistic regression takes non convex form but doesn't in linear regression? [duplicate]

I am taking this deep learning course from Andrew NG. In the 3rd lecture of 2nd week of the first course, he mentions that we can use RMSE for logistic regression as well but it will take a nonconvex ...
0
votes
1answer
23 views

Penalization term for unfairness

I am reading [1], where the researchers do a logistic regression, but add to the loss function the following penalization term for fairness $ R^{AVD}_{FP}(\theta; S) = \left\lvert \dfrac{\sum\limits_{...
5
votes
0answers
60 views

Asymmetric or unequal misclassification costs in random forest

I have a general question about asymmetric costs. In machine learning problems, there are times when the cost of a false positive is different from the cost of a false negative. Accordingly, models ...
1
vote
1answer
31 views

Enforcing Dirac delta-like Activations on a Neural Network

I am working on a custom neural network model including convolutional and dense layers. I intend to enforce outputs a certain dense layer to approximate a Dirac delta function (or any localized pulse)....
0
votes
1answer
99 views

why is VAE reconstruction loss equal to MSE loss

At which situations does reconstruction loss of VAE equals MSE loss between input and reconstructed output? Other answers where not complete!
1
vote
0answers
22 views

What is the relationship between minimizing prediciton error versus parameter estimation error?

With the advent of statistical learning techniques, people are talking a lot about prediction error, while in classical statistics, one is focusing on parameter estimation error. What is the ...
1
vote
0answers
43 views

What is the relation between a loss function and an energy function?

A loss function is a function that measures the distance between the expected value and the actual value of a model (an example of a loss function is the cross entropy). An energy function can be ...
0
votes
0answers
5 views

Ranking criterion vs. entropy criterion

Problem In a classical NLP paper (Natural language processing (almost) from scratch) I am reading now, the authors claim that The entropy criterion lacks dynamical range because its numerical ...
0
votes
0answers
34 views

relation among loss function / MLE / Bayesian estimation

I have read a lot of stuff on the relation between minimizing a loss function / maximizing the likelihood / choose a centrality measure of the posterior (Bayesian estimation); but I cannot see a clear ...
0
votes
0answers
8 views

Probability of Incurring Maximum Loss

In online classification one can use mistake bound learning, where one assumes that all $y$ are generated by some target mapping $h^*: \mathcal{X} \rightarrow \mathcal{Y},\,\, h^* \in \mathcal{H}$. ...
1
vote
0answers
24 views

What could cause a flat loss function to suddenly decrease in a u-net used for denoising?

So I am trying to understand U-Nets better, and I built a very shallow U-Net and trained it to denoise MNIST images (training set is 90% of the whole dataset). The loss function evolution I obtained ...
2
votes
1answer
35 views

Constant validation loss and increasing validation accuracy

I am training a fully convolutional network. The loss is decreasing whilst the validation loss stays mostly where it is. There is some variance in the validation loss. I thought it might overfits, ...
1
vote
0answers
12 views

Different classification loss for K-nearest neighbours

Suppose we have a general classification loss instead of a 0-1 loss. How can we modify k-NN to accommodate such a loss function? I thought about using a weighted loss matrix where $L(i,j)=0$ when $i=...