Questions tagged [loss-functions]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

How to use cosine similarity within triplet loss

The triplet loss is defined as follows: $$ L(A, P, N) = max(‖f(A) - f(P)‖² - ‖f(A) - f(N)‖² + margin, 0) $$ where $A$=anchor, $P$=positive, and $N$=negative are the data samples in the loss, and $...
user avatar
  • 721
0 votes
0 answers
18 views

What are some good relative regression metrics?

Which relative regression metrics exist? What are their strengths and weaknesses? In what case do you use each? --> Bonus point if they are already/easily implemented in Scikit-Learn. I have a ...
user avatar
0 votes
1 answer
20 views

Training loss after last epoch differs from training loss (same data!) during evaluation

I am building a deep convolutional model with a custom loss function. As a first step, I am trying to bring training loss down as far as possible to see if my model can overfit. Training with on only ...
user avatar
1 vote
1 answer
21 views

Can Poisson deviance be used to evaluate models that use loss functions other than Poisson? (Such as MSE)

I am currently doing a a study on emergency department utilization rates at various geography levels. Especially of interest, are tree-based approaches to this analysis - namely random forest and GBMs....
user avatar
  • 101
1 vote
1 answer
42 views

Time-series loss function combining RMSE and Classification accuracy

I am working on a time series regression problem applied to finance. I am interested in predicting the price change of a stock and how much will it change by. I have framed this as a regression ...
user avatar
0 votes
0 answers
18 views

GoogleNet-LSTM, cross entropy loss does not decrease [duplicate]

...
user avatar
  • 1
0 votes
0 answers
31 views

Classifier as loss function for image generation, "tricked" too easily

I have a StyleGAN model for which I want to manipulate some properties of the generated image, e.g. eyes being opened/closed [1]. I finetuned a classifier (using pretrained ConvNeXt) on subset of 500 ...
user avatar
0 votes
0 answers
30 views

Finding coefficients for a loss function with multiple parts and different scales

I'm working on reusing the paper Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation, which deforms a 3D mesh to make it fit some 2D images, in another scenario than the original one. The loss ...
user avatar
0 votes
0 answers
12 views

Can I adjust the Wasserstein GAN loss function for my particular data?

I am working on building Generative Adversarial Networks for the purpose of generating synthetic flight data. The GAN will be trained on actual time-series flight data in the form of a (n,m,9) array ...
user avatar
0 votes
0 answers
16 views

How do implement a cross corellation as loss function?

Hi I would like to use the normalized crosscorrelation coefficient NCC as a loss function in order to compare a output matrix A with a reference matrix B. NCC=Sum_{ij} (A_{ij}-)(B_{ij}-)/(||A||*||B||) ...
user avatar
0 votes
0 answers
8 views

How to set a manual loss for the generator in a GAN?

I have a trained GAN model that generates images. I want to replace the discriminator with a human. Ideally, this person enters a score of how much he or she likes the images and that is the loss for ...
user avatar
  • 101
3 votes
3 answers
68 views

How to find the gradient when a black box I/O function is involved in evaluation of the loss?

I am trying to learn a neural network $NN_\pi$ to minimize the loss function $$ L_{\pi} = || Y_{true} - F(X_{true}, NN_{\pi}(X_{true}) ) ||^2 $$ where $F$ is a black box (I/O) function (we only ...
user avatar
0 votes
0 answers
8 views

standard linear model vs linear model with an artificial restriction on the coefficient effects

Consider a standard linear model and one with an artificial restriction on the coefficient effects, where k is a fixed regularization parameter. Standard: $y = b_1 x_1 + b_2 x_2 + b_3 x_3$ Restricted: ...
user avatar
  • 51
15 votes
2 answers
806 views

Why not use evaluation metrics as the loss function?

Most algorithms use their own loss function for optimization. But these loss functions are always different from metrics used for actual evaluation. For example, for building binary classification ...
user avatar
  • 815
0 votes
0 answers
12 views

How to create a loss function that penalizes duplicate indices in the output tensor?

We're working on a sequence-to-sequence problem using pytorch, and are using cross-entropy to calculate the loss when comparing the output sequence to the target sequence. This works fine and ...
user avatar
  • 1,121
1 vote
0 answers
19 views

Capping labels negatively impacts business metrics

I have this deep neural network model with an integer label to predict. The label is heavily skewed so we cap the labels at some value (let's say 90 %ile). Now when we build and run the model, it ...
user avatar
  • 279
1 vote
0 answers
22 views

Predicting estimated time of arrival (ETA) with custom loss function

I have an assignment of ETA prediction that I'd like to seek some advice from you. I need to train a model from a dataset to predict ETA of parcels as dependent variable, which is discrete (in days). ...
user avatar
  • 11
0 votes
0 answers
6 views

I have troubles with getting a neural network to learn a function [duplicate]

I am currently trying to make an artificial neural network learn a function that takes a 1 dimensional input and the output is also of dimension 1. However, I have some troubles making the network ...
user avatar
0 votes
0 answers
41 views

Impact of L1 and L2 regularisation with cross-entropy loss

When we are dealing with Mean Square Error (MSE) loss function in optimization problems, we often add $L_1$ or $L_2$ penalty terms (or a combination of both) to the MSE loss function while training. ...
user avatar
0 votes
0 answers
32 views

loss function for neural network with matrix output

I want to train a neural network with $\mathbb{R}^{d\times d}$ symmetric and positive definit matrix output. Are there rules which loss function ( in dependence of the predicted matrix $A_{\text{pred}}...
user avatar
0 votes
0 answers
43 views

Advarsarial autoencoder loss function - Using MSE and BCE both

I came across this implementation of AAE on financial data to detect anomalies https://github.com/GitiHubi/deepAD/blob/master/KDD_2019_Lab.ipynb. In here for the VAE part of AAE, the author is using ...
user avatar
1 vote
0 answers
64 views

YOLO v2 loss function

I'm trying to understand (and implement) the YOLOv2 loss function, which is not given explicitly in the original paper. There are several posts on this topic, but quite a few seem to confuse the ...
user avatar
  • 11
0 votes
0 answers
64 views

How can I get the Binary Cross Entropy from the Cross Entropy function for GANs

I got the definition of log-likelihood by Goodfellow's Deep Learning book: \begin{equation} \label{eq:loglikelihood} \theta_{ML} = {argmax}\sum_{i=1}^{m} \log p_{model}(x_i; \theta). \end{...
user avatar
2 votes
1 answer
89 views

Correct loss function and metric for regression of count data in neural network

I am using a convolutional neural network to predict the number of occurrences of a certain pattern in time series data. Since there might be potentially any count of such patterns in a time series, I ...
user avatar
1 vote
0 answers
24 views

What is a differentiable approximation to the indicator function a != b [closed]

I am dealing with an optimization problem where I'd like to regularize two parameters $a$ and $b$. The penalty should be $1$ if the parameters differ and $0$ if they are the same. The motivation is ...
user avatar
0 votes
0 answers
7 views

Validation Loss for my binary image classifier model is increasing. how to bring it down? [duplicate]

I am new to the domain of Deep learning and I have been trying to create a binary image classifier using a dataset which I created by myself. I am building the model from scratch. It is CNN model. ...
user avatar
  • 1
1 vote
0 answers
11 views

Design loss function for model-based reinforcement learning

I'm doing some model-based reinforcement learning, and I'm stuck at how to better design the loss function for fitting the dynamic model of the environment. In continuous state and action space, the ...
user avatar
  • 1,653
2 votes
1 answer
133 views

Does scoring rules really only apply to categorical outcomes?

The wikipedia article on scoring rule says that It is applicable to tasks in which predictions must assign probabilities to a set of mutually exclusive outcomes or classes. The set of possible ...
user avatar
0 votes
0 answers
13 views

implementing a neural network U-Net with imbalanced classes, implementing the loss function

my problem is : i have a neural network U-Net, but to do the segmentation on my sparse annotation, i need to implement the loss function for the imbalanced classes so the article says, that there is a ...
user avatar
0 votes
0 answers
13 views

Can we compare rigorously the computing time to evaluate ReLU or other nonlinear smooth activations?

Can we say that, independently of the computer, computing relu and relu' is cheaper than computing f and f' for some other smooth non-linear activation (e.g. logistic, tanh)? If not, what are the ...
user avatar
0 votes
0 answers
32 views

weighted maximum likelihood as loss function

I have built a little neural network that I use for regression. ...
user avatar
  • 113
2 votes
1 answer
47 views

Why is cross entropy loss better than MSE for multi-class classification? [duplicate]

I know there's a lot of material on this, but I'm still struggling to find a scenario where cross-entropy loss is better than MSE loss for a multi-class classification problem. For example, if we have ...
user avatar
  • 51
1 vote
1 answer
90 views

Why does contrastive loss distinguish positive from negative samples?

Trying to learn Siamese networks for ranking tasks from here, I find it hard to understand why the contrastive loss is not symmetric for positive and for negative examples. The contrastive loss $L(A, ...
user avatar
  • 373
2 votes
0 answers
46 views

Why does cotrastive loss and triplet loss have the margin element in them?

Trying to learn Siamese networks for ranking tasks from here, I find it hard to understand why triplet loss and contrastive losses have the "margin" element in them. The contrastive loss $L(...
user avatar
  • 373
1 vote
0 answers
47 views

Why does triplet loss outperform contrastive loss?

Trying to learn Siamese networks for ranking tasks from here, I find it hard to understand why triplet loss was ever introduced at all, theoretically. I understand it works better in practice, but ...
user avatar
  • 373
1 vote
0 answers
16 views

Tensorflow - calling a model inside a GradientTape scope VS calling it inside a loss function

Is there a difference in the gradient computation between the two code snippets ... Code 1: ...
user avatar
0 votes
1 answer
37 views

Expected Prediction Error for 0-1 Loss Function

In ESL on pages 20 and 21, we have a derivation of expected prediction error of a classification rule $\hat{G}(X)$: $$ EPE(\hat{G}) = E_X\sum_{k=1}^{K}L[\mathcal{G}_k, \hat{G}(X)]P(\mathcal{G}_k|X) $$ ...
user avatar
1 vote
1 answer
35 views

Loss surface visualisation/intuition

I'm trying to wrap my head around a loss surface in pytorch. This is for work, not a homework assignment. let's say we have a model y = model(x) error = y - y_label ...
user avatar
0 votes
0 answers
23 views

Loss Size Index Function of A Gamma Random Variable

I'm trying to prove that the loss size index function of a Random Variable Y, which is distributed as a Gamma Random Variable ($Y \sim Γ(γ,c)$) has the following expression: $$ I(y) = \frac{\textit{G}(...
user avatar
1 vote
1 answer
75 views

Learned Loss Attenuation for Classification

In the paper What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? they propose loss functions that capture aleatoric uncertainty. My question heavily relies on understanding of ...
user avatar
  • 13
0 votes
0 answers
11 views

Minimize risk and add rejection to model

I want to minimize the risk of a Gaussian model with a cost for false negatives and false positives. The model uses Naive Bayes algorithm and solves a binary classification problem: $$P(x_i \mid y) = \...
user avatar
  • 1
9 votes
3 answers
626 views

Consistency between two outputs of a neural network

I'm trying to fit a dense neural network based on tabular data input, where the outputs are two separate classification vectors, with one cross-entropy loss function for each. Example: given a few ...
user avatar
  • 529
0 votes
0 answers
9 views

Interpreting validation loss and accuracy for various learning rates

I am having a hard time comparing the effect of different learning rates on validation loss and accuracy. Would I be right in assuming that a Learning rate of 0.0001 was the most successful as the ...
user avatar
0 votes
1 answer
26 views

Function Cost Neural Network [closed]

Neural networks use cost functions to minimize error and make the model better for supervised models. Example: Regression cost Function: Regression models deal with predicting a continuous value for ...
user avatar
  • 116
2 votes
1 answer
24 views

Deriving Squared Loss Function from Normality Assumption of Output and Likelihood of Parameter

This question will seem very beginner in this forum, but I'm indeed a beginner. I am attempting to understand method of least square for regression. So, likelihood of parameter is defined as $$\...
user avatar
0 votes
0 answers
21 views

Loss Function for Polar Coordinates Target Variables?

Suppose I want to do regression on targets that are polar coordinates. What loss functions are appropriate? I know I could always use mean squared error, but I'm not sure that's the most appropriate.
user avatar
4 votes
2 answers
229 views

Is there a preference in the regression performance metric for regression models with the same type of loss minimization?

I applied two regression models (ordinary least square (OLS) and linear absolute regression) to the same dataset, where this dataset is split into train and test sets. Two performance measures are ...
user avatar
  • 1,487
0 votes
0 answers
16 views

Loss Jumps while Train a Fully Connected MLP

i am currently trying to train a Fully Connected MLP with vibration data from a machine aggregate for classification. During training, the loss jumps up abruptly in each epoch. Here is an excerpt of ...
user avatar
2 votes
2 answers
98 views

Why use regularization instead of feature selection for logistic regression? [duplicate]

For a non-linearly separable problem, when there are enough features, we can make the data linearly separable. It seems to me that for logistic regression, the reason of overfitting is always ...
user avatar
3 votes
1 answer
29 views

loss bounded below when using MSE regardless of the model I choose, is this normal?

I've been experimenting with regression models and when checking the loss of some models (MLP, RNN, CNN) I constructed using MSE, the evaluation loss is bounded below by 1.1331, and the training loss ...
user avatar

1
2 3 4 5
21