A function used to quantify the difference between observed data and predicted values according to a model. Minimization of loss functions is a way to estimate the parameters of the model.

learn more… | top users | synonyms

0
votes
0answers
8 views

Loss function selection for weighting errors differently

I am building a regression model where I want to score/optimize/train 'over-predictions' to be twice costly as under predictions. I am attempting to do this in R and hopefully with caret package. ...
0
votes
0answers
9 views

Intuition for Normalized Squared Loss error function?

In terms of optimization squared loss is perhaps the most common error function used for regression. I've seen another function named "Normalized Squared Loss" mentioned, described as The ...
0
votes
0answers
6 views

Weights and hinge loss/linear SVM

Say I have a dataset with this distribution Class A: 10 Examples Class B: 100 Examples Class C: 1000 Examples Hence, i am trying to build a classifier using linear SVM. Baring all concerns about ...
0
votes
0answers
46 views

Loss function for multiple predictions without ground truth

I'm searching for a loss function to determine the difference between two predictions made by several multi-label classifiers. Consider the following: Prediction of classifier A is [0.2,0.5,0.1,0.8] ...
0
votes
0answers
24 views

Loss Function for Multinomial Logistic Regression - Cannot find its derivative

For Multinomial Logistic Regression we can define the Loss Function in the following way: $J(\theta)=\frac{-1}{m}\sum\limits_{i=1}^m\sum\limits_{j=1}^k ...
0
votes
1answer
37 views

Predicting values from linear regression

I've got a data set consisting of olympic years and the winning times for the womens 100m. I can plot a line throught the data using matlab as such: ...
1
vote
1answer
19 views

How do I show that the mean of the posterior density minimizes this squared error loss function?

This exercise comes from Koop's Bayesian Econometrics. Given $\theta$, the parameter(s) of a model (in this case $\theta$ is a scalar), $\tilde{\theta}$, the point estimate of $\theta$, and constants ...
0
votes
0answers
14 views

Deducing the regression function using the Squared Error Loss Function [duplicate]

I am reading Elements of Statistical Learning, and came across a deduction which I cannot understand. In the second chapter, the author defines the squared error loss and deduces the conditional ...
3
votes
0answers
33 views

Bias-variance decomposition

In section 3.2 of Bishop's Pattern Recognition and Machine Learning, he discusses the bias-variance decomposition, stating that for a squared loss function, the expected loss can be decomposed into a ...
0
votes
0answers
16 views

Perceptron Inseparable Case - Hinge loss

I am trying to understand the case where the perceptron algorithm can't find a perfect seperator. I am trying to understand how we bound the number of mistakes by using the hinge loss. My intuition ...
0
votes
1answer
8 views

How to target data gathering to minimize loss-function?

I have a data-set, a model (single variable) and a loss function. I can collect more data but each data point requires significant analysis to obtain. Hence how can I target the data collection to ...
0
votes
0answers
31 views

Deriving the gradient of a loss function for generalized logistic regression

I am trying, without much success so far, to derive the gradient of the following cost function in order to fit a logistic curve to some data: $J(a, k, b, m) = \sum_i^n(y_i - a + \frac{k - a}{(1 + ...
0
votes
0answers
28 views

What loss function should one use to get a high precision or high recall binary classifier?

I'm trying to make a detector of objects that occur very rarely (in images), planning to use a CNN binary classifier applied in a sliding/resized window. I've constructed balanced 1:1 ...
1
vote
1answer
18 views

Looking for an error measure like MAE that weights overprediction more than underprediction

I am trying to evaluate my prediction results using common error measures like the MAE, MSE or RMSE. For me it is much worse if the predicted value is higher than the true value. If it is less, it is ...
3
votes
0answers
31 views

Selection of Loss Function for Weather Insurance

I am using machine learning to predict agricultural yields using weather variables as inputs. I have been thinking about what loss function to optimize. I have been using RMSE thus far. The ...
0
votes
1answer
41 views

How can logistic loss return 1 for x = 0?

I have looked at the logistic loss function at many different sources, and many places I find it plotted like shown here: Taken from ...
3
votes
2answers
54 views

What loss function should I use for binary detection in face/non-face detection in CNN?

I want to use deep learning to train a face/non-face binary detection, what loss should I use, I think it is SigmoidCrossEntropyLoss or Hinge-lossenter link description here is that right, but I also ...
0
votes
0answers
12 views

Assessing a vector of errors in modeling

The quality of a model is often assessed based on a figure of merit such as RMSE. This reduces the individual errors in the model to a single number without assessing the errors as a population of ...
0
votes
0answers
52 views

Siamese networks and backpropagation

I have a misunderstanding how backpropagation works in siamese networks. Here is similar topic but it doesn't cover my question. How does the back-propagation work in a siamese neural network? ...
0
votes
0answers
26 views

Hinge loss in siamese network

I'm trying to implement method of comparing images' patches using siamese neural networks described here http://arxiv.org/pdf/1510.05970v1.pdf Authors propose using siamese CNN and they define hinge ...
7
votes
1answer
93 views

Constructing a problem-specific loss function

Problem Description I'm beginning network construction for a problem that I feel could have a far more insightful loss function than a simple MSE regression. My problem deals with multi-category ...
0
votes
0answers
44 views

Confusion in Training Logistic Regression Model

I have a confusion in Cost function in Logistic Regression, I have a set of data ...
1
vote
1answer
40 views

Determine the best sample size for minimum expected loss

Let $\theta \sim Gamma(1,2)$ and $X_1,...,X_n$ iid such that $X_i|\theta \sim Poisson(\theta)$. It is asked to determine the best sample size $n^*$ such that the posteriori risk $$L(\theta, d) = ...
3
votes
1answer
33 views

Quantreg : Unbalanced residuals

I'm trying to use the quantreg package to fit an exponential curve. Here is a reproductible example. IRL I have much more complex data with outliers, that's why I ...
3
votes
4answers
62 views

Logistic regression loss with continuous labels

I want to learn a logistic regression model but my outcome variable can take continuous values in [0;1], not only binary labels {0;1}. My model thus still needs a logistic link (to bound my ...
3
votes
0answers
72 views

CNN - Extract visual information via Gradient Descent with Backpropagation

I'm trying to reproduce the results from this paper: Mahendran, Aravindh, and Andrea Vedaldi. "Understanding deep image representations by inverting them." arXiv preprint arXiv:1412.0035 (2014). One ...
3
votes
1answer
118 views

Quadratic loss function implying conditional expectation

I am reading Bishop's pattern recognition book. In the decision theory part he first derives that using a quadratic loss function implies that our estimate $y(x)$ should be the conditional expectation ...
0
votes
0answers
52 views

Expected loss function in matlab

I have two matrices: The true one and the approximated one. In fact I want to compute the expected loss of the difference, that is, $E[||\Sigma_{approximated} - \Sigma_{true}||]$. In matlab, I ...
0
votes
0answers
39 views

Hinge loss values SVM interpretation

I'm using WEKA SPegasos SVM algorithm with Hinge Loss and I'm getting such Hinge Loss values. How to interpret those values ?? Are the values with the biggest absolute value the most significiant ?? ...
0
votes
0answers
43 views

Bayesian Decision Theory: Expected Loss

This question involves Bayesian decision theory and I have to try and derive the expected loss function. I'm not sure how to even approach this, so any help on how to begin this would be very much ...
2
votes
1answer
20 views

Struggling with payoff matrix

I've been struggling finding the loss functions, $L(\theta,d_1)$ and $L(\theta,d_2)$, for the following question: Items I manufacture are either independently flawed with probability, $p$, or ...
1
vote
0answers
35 views

Loss Size Index Function of A Lognormal Random Variable

I have this tutorial question and I've gone through the solutions, getting all but one line of working. I broke down the question to this point but I can't seem to get out the following. So Loss Size ...
0
votes
0answers
33 views

Changing optimization algorithm while optimizing

I am currently training some convolutional neural networks with cross-entropy loss. Thus, the function I am optimizing is non-convex, and at the moment I am using an optimization algorithm called Adam ...
1
vote
1answer
80 views

How can I force my model to predict the samples that are close to zero?

I have a large amount of inventory data and I am trying to predict when the inventory gets low using one component of the change in inventory (yes I know this doesn't describe inventory very well by ...
0
votes
0answers
44 views

derive loss function for gamma regression

In the R package mboost there is a family called "GammaReg" which implementes "negative Gamma log-likelihood with logarithmic link function". Still, I don't really ...
3
votes
1answer
68 views

Shrinkage of the Sample Covariance matrix

Assume we have N independent and identically distributed random vectors $X_1, X_2, ..., X_N$ where each of them is of size p $\times$ 1. The sample covariance matrix, denoted here by $S$, is computed ...
3
votes
1answer
151 views

What's the measure to assess the binary classification accuracy for imbalanced data?

Now I have binary classification problem with positive samples roughly 100 times the number of negative samples. In this case the normal accuracy measure (predict == label) is not a good measure. What ...
0
votes
1answer
54 views

Why linear regression one variable uses squared cost function? [duplicate]

This is about linear regression course given by Andrew Ng on Coursea about machine learning. why cost function is $$ \frac{1}{m} \sum _{i=1}^m \left(h_\theta(X^{(i)})-Y^{(i)}\right)^2 $$ and not ...
1
vote
1answer
63 views

derivative of loss function

If we have a paralyzed loss function of the form of: \begin{align} L(\beta)& =\frac{1}{2}(y-X\beta)^T(y-X\beta)+ \lambda \beta^T f(\beta) \end{align} where $X_{n\times m}$ and $\beta_{m \times ...
2
votes
1answer
376 views

Scikit Binomial Deviance Loss Function

This is scikit GradientBoosting's binomial deviance loss function, ...
1
vote
3answers
196 views

Best loss function for very sparse real-valued data

Suppose the target output of my data prediction model is an $M\times N$ matrix where $95\%$ of the values are $0.0$ and the other values are anywhere between $0.0$ and $1.0$, what would be a good loss ...
0
votes
1answer
190 views

Gradient for hinge loss multiclass

I am little confused when trying to find the gradient for the multiclass hinge loss: $l(y) = \max( 0, 1 + \underset{r \neq y_i}{ \max } W_r \cdot x_i - W_{y_i} \cdot x_i)$ Where $W^{k \times n}$ is ...
2
votes
0answers
270 views

Loss and dropout in deep learning

I have a CNN with 3 convolutional layers, 1 max-pooling layer and 2 fully-connected layers before applying softmax classification. The CNN is trained with Adagrad and I achieve a quite good ...
0
votes
0answers
26 views

Logistic loss approximation

In many implementations of logistic loss (example sklearn) I see the following code(adapted from sklearn), where p is the prediction and y the true value: ...
1
vote
1answer
338 views

Gradient of loss function for (non)-linear prediction functions

$ \newcommand{\y}{\mathbf{y}} \newcommand{\wv}{\mathbf{w}} \newcommand{\xv}{\mathbf{x}} \newcommand{\loss}{L(\wv;\xv, y)} $ I'm trying to clear up the calculation of the gradient of a loss function, ...
0
votes
0answers
85 views

Meaning of the prior and loss parameters in rpart in R

Could someone please explain to me what specifying priors and/or loss parameters in R's rpart actually do? I found R's documentation completely unhelpful. For example, let's suppose I have a highly ...
1
vote
0answers
81 views

hinge loss vs logistic loss advantages and disadvantages/limitations

Hinge loss can be defined using $\text{max}(0, 1-y_i\mathbf{w}^T\mathbf{x}_i)$ and the log loss can be defined as $\text{log}(1 + \exp(-y_i\mathbf{w}^T\mathbf{x}_i))$ I have the following questions: ...
0
votes
0answers
38 views

maximum mis-classification loss of hinge loss

In the plots and in some lecture notes, I read that hinge loss is bounded between (0,2). But I can not understand that. By definition, hinge loss is (standard one) ...
4
votes
1answer
67 views

showing that $\bar{X}$ is inadmissible by comparing with $\max(\bar{X},2)$ under squared error loss function

suppose $X_1,X_2,\ldots,X_n$ be a random sample of $N(\theta,1), \theta>2$. how can I show $\bar{X}$ is inadmissible estimator Compared to $\max(\bar{X},2)$ under Squared error loss function
3
votes
1answer
65 views

Does maximum likelihood minimize a kind of generalized “0-1 loss”?

A very good point was raised here about how the optimal betting strategy under 0-1 loss was to bet on the mode, while under MSE loss the optimal strategy was to bet on the mean. Maximum likelihood ...