# Questions tagged [loss-functions]

A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

1,014 questions
Filter by
Sorted by
Tagged with
13 views

43 views

### Advarsarial autoencoder loss function - Using MSE and BCE both

I came across this implementation of AAE on financial data to detect anomalies https://github.com/GitiHubi/deepAD/blob/master/KDD_2019_Lab.ipynb. In here for the VAE part of AAE, the author is using ...
1 vote
64 views

### YOLO v2 loss function

I'm trying to understand (and implement) the YOLOv2 loss function, which is not given explicitly in the original paper. There are several posts on this topic, but quite a few seem to confuse the ...
64 views

### How can I get the Binary Cross Entropy from the Cross Entropy function for GANs

I got the definition of log-likelihood by Goodfellow's Deep Learning book: \begin{equation} \label{eq:loglikelihood} \theta_{ML} = {argmax}\sum_{i=1}^{m} \log p_{model}(x_i; \theta). \end{...
89 views

### Correct loss function and metric for regression of count data in neural network

I am using a convolutional neural network to predict the number of occurrences of a certain pattern in time series data. Since there might be potentially any count of such patterns in a time series, I ...
1 vote
24 views

### What is a differentiable approximation to the indicator function a != b [closed]

I am dealing with an optimization problem where I'd like to regularize two parameters $a$ and $b$. The penalty should be $1$ if the parameters differ and $0$ if they are the same. The motivation is ...
7 views

### Validation Loss for my binary image classifier model is increasing. how to bring it down? [duplicate]

I am new to the domain of Deep learning and I have been trying to create a binary image classifier using a dataset which I created by myself. I am building the model from scratch. It is CNN model. ...
1 vote
11 views

### Design loss function for model-based reinforcement learning

I'm doing some model-based reinforcement learning, and I'm stuck at how to better design the loss function for fitting the dynamic model of the environment. In continuous state and action space, the ...
133 views

### Does scoring rules really only apply to categorical outcomes?

The wikipedia article on scoring rule says that It is applicable to tasks in which predictions must assign probabilities to a set of mutually exclusive outcomes or classes. The set of possible ...
13 views

### implementing a neural network U-Net with imbalanced classes, implementing the loss function

my problem is : i have a neural network U-Net, but to do the segmentation on my sparse annotation, i need to implement the loss function for the imbalanced classes so the article says, that there is a ...
13 views

### Can we compare rigorously the computing time to evaluate ReLU or other nonlinear smooth activations?

Can we say that, independently of the computer, computing relu and relu' is cheaper than computing f and f' for some other smooth non-linear activation (e.g. logistic, tanh)? If not, what are the ...
32 views

### weighted maximum likelihood as loss function

I have built a little neural network that I use for regression. ...
47 views

### Why is cross entropy loss better than MSE for multi-class classification? [duplicate]

I know there's a lot of material on this, but I'm still struggling to find a scenario where cross-entropy loss is better than MSE loss for a multi-class classification problem. For example, if we have ...
1 vote
90 views

1 vote
47 views

### Why does triplet loss outperform contrastive loss?

Trying to learn Siamese networks for ranking tasks from here, I find it hard to understand why triplet loss was ever introduced at all, theoretically. I understand it works better in practice, but ...
1 vote
16 views

### Tensorflow - calling a model inside a GradientTape scope VS calling it inside a loss function

Is there a difference in the gradient computation between the two code snippets ... Code 1: ...
37 views

### Expected Prediction Error for 0-1 Loss Function

In ESL on pages 20 and 21, we have a derivation of expected prediction error of a classification rule $\hat{G}(X)$: $$EPE(\hat{G}) = E_X\sum_{k=1}^{K}L[\mathcal{G}_k, \hat{G}(X)]P(\mathcal{G}_k|X)$$ ...
1 vote
35 views

### Loss surface visualisation/intuition

I'm trying to wrap my head around a loss surface in pytorch. This is for work, not a homework assignment. let's say we have a model y = model(x) error = y - y_label ...
23 views

626 views

### Consistency between two outputs of a neural network

I'm trying to fit a dense neural network based on tabular data input, where the outputs are two separate classification vectors, with one cross-entropy loss function for each. Example: given a few ...
9 views

### Interpreting validation loss and accuracy for various learning rates

I am having a hard time comparing the effect of different learning rates on validation loss and accuracy. Would I be right in assuming that a Learning rate of 0.0001 was the most successful as the ...
26 views

### Function Cost Neural Network [closed]

Neural networks use cost functions to minimize error and make the model better for supervised models. Example: Regression cost Function: Regression models deal with predicting a continuous value for ...
24 views

### Deriving Squared Loss Function from Normality Assumption of Output and Likelihood of Parameter

This question will seem very beginner in this forum, but I'm indeed a beginner. I am attempting to understand method of least square for regression. So, likelihood of parameter is defined as \...
21 views

### Loss Function for Polar Coordinates Target Variables?

Suppose I want to do regression on targets that are polar coordinates. What loss functions are appropriate? I know I could always use mean squared error, but I'm not sure that's the most appropriate.
229 views

### Is there a preference in the regression performance metric for regression models with the same type of loss minimization?

I applied two regression models (ordinary least square (OLS) and linear absolute regression) to the same dataset, where this dataset is split into train and test sets. Two performance measures are ...
16 views

### Loss Jumps while Train a Fully Connected MLP

i am currently trying to train a Fully Connected MLP with vibration data from a machine aggregate for classification. During training, the loss jumps up abruptly in each epoch. Here is an excerpt of ...