The loss-functions tag has no wiki summary.
0
votes
1answer
62 views
Minimizing the expected loss
I was wondering about the motivation behind the following definition of expected loss:
$$E[L] = \sum_{k} \sum_{j} \int_{R_{j}} L_{kj} p(x, C_{k})dx$$
where $L_{kj}$ is the loss matrix, in which $j$ ...
0
votes
0answers
45 views
Fitting a logistic curve with absolute value loss
I was wondering whether the procedure of fitting a logistic curve with absolute value loss has a well known name / commonly available implementation.
The idea behind this is that if I have two ...
2
votes
2answers
88 views
Minimizing variance of an estimator under sampling cost penalty
I have an estimator $t$, whose variance depends on the dimension of
my sample $x_{1:n}$:
$$
\text{Var}(t(x_{1:n})) = f(n).
$$
Suppose that the form of $f(n)$ is known. I would like to determine
what ...
1
vote
0answers
56 views
A “Gambler's Loss function”?
What is a good loss function for a predictive model used by gamblers?
I've been reading a bit about loss functions recently. I've always just went with MSE (e.g., for a couple of neural network ...
3
votes
1answer
81 views
Expectation notations
In Statistical Decision Theory, one often studies the following two measures (from "The Bayesian Choice"):
Average loss (aka the frequentist risk):
$R\left(\theta,\delta\right) = ...
1
vote
0answers
39 views
Representer theorem for vector-valued functions
Is there a representer theorem for loss-functions of the form $\sum_{i}(f(x_i \mathbb{.}),y_i)$ of the form where the output of $f(.)$ is a vector and the domain is also a vector. Also, there is a ...
1
vote
0answers
55 views
Can I define an $R^2$-like measure in this way when predicting with exotic loss functions?
It is common in regression to see $R^2$ formulated as follows: $$R^2\equiv 1 - {SS_{\rm err}\over SS_{\rm tot}},$$ where $SS_\text{err}=\sum_i (y_i - f_i)^2$ and $SS_\text{tot}=\sum_i ...
2
votes
0answers
68 views
Notation in GBM package vignette: expected value of loss functions
Can anyone help with the understanding of this notation (and idea) from the vignette for GBM in R?
It starts with the following:
Question 1: I believe this is simply saying that we are looking for ...
11
votes
2answers
249 views
How to design and implement an asymmetric loss function for regression?
Problem
In regression one usually computes the mean squared error (MSE) for a sample:
$$
\text{MSE} = \frac{1}{n} \sum_{i=1}^n\left(g(x_i) - \widehat{g}(x_i)\right)^2
$$
to measure the quality of a ...
5
votes
1answer
373 views
L1 regression estimates median whereas L2 regression estimates mean?
So I was asked a question on which central measures L1 (i.e., lasso) and L2 (i.e., ridge regression) estimated. The answer is L1=median and L2=mean. Is there any type of intuitive reasoning to this? ...
2
votes
0answers
88 views
Asymptotics of 0-1 classification loss
I am interested in training a simple binary linear classifier. That is, I will find a vector of weights $\bf w$ such that I can predict the class of new example by the sign of
$f(x) = w^T x$.
I ...
2
votes
0answers
134 views
Cramer-Rao type bound for Information Gain
I am interested in the Bayes risk of some distribution $\pi$
$$
r(\pi) = \mathbb{E}_{\pi(x)}[ \mathbb{E}_{\Pr(y|d,x)}[L(x,\hat x(y|d))]],
$$
where $L$ is some loss function and $\hat x$ is the ...
2
votes
1answer
59 views
Convexity of loss function with respect to the mean
Let $X\,$ be a non-negative r.v. with known pdf $f(x|\theta)$ but with a single unknown parameter $\theta$. Suppose that the mean $\mu$ can be used to uniquely determine the value of $\theta$, i.e. if ...
0
votes
1answer
239 views
Changing logistic regression's loss function
We're using logistic regression to predict events probability. Logistic regression tries to minimize the residual variance (sum of squared residuals). However, in our specific problem we would like ...
2
votes
0answers
104 views
Does cross validation work with asymmetric loss functions?
My simple question is does cross validation work with an asymmetric loss function?
I cannot find docs on google to answer.
2
votes
3answers
183 views
Multiclass classification when class distribution is known
What is an example of an algorithm that, when i have a known distribution across discrete groups and I have some sort of model score that a person is in each group, assigns persons to groups such that ...
1
vote
1answer
81 views
Weighted loss function for non-random sample
When comparing a regression estimation method (Y vs X) I currently use a weighted squared loss function:
$$
\int_{-\infty}^{\infty}(\hat{f}(x)-f(x))^2 \, \hat{p}(x) \, dx
$$
Where $\hat{f}(x)$ ...
1
vote
2answers
202 views
Matching loss function for tanh units in a neural net
There's not much more I can add to the question. Googling has mostly turned up research papers on springerlink and other sites I don't have access to.
Given a neural network model with $tanh(x)$ as ...
3
votes
1answer
314 views
Dual problem for L2 support vector machine
Here is the dual problem for L2 support vector machine:
$$\max_{\alpha\in\mathbb{R}^{n}} 2\alpha^{T}y-\alpha^{T}\left(K+n\lambda Id_{\mathbb{R}^{n}}\right)\alpha$$
$$\forall i\in\left\{ ...
3
votes
0answers
220 views
Normalized sigmoid loss function for boosting?
It seems non-convexity of loss function is not such a problem for boosting with a normalized sigmoid loss function. Do you know any further work showing better results with this kind of boosting than ...
5
votes
2answers
243 views
Hinge loss with one-vs-all classifier
I'm currently looking at the unconstrained primal form of the one-vs-all classifier
$$\sum\limits_{i=1}^{N_I} \sum\limits_{k=1,\atop k \neq y_i}^{N_K} L(1+ ...
5
votes
3answers
462 views
Gradient descent oscillating a lot. Have I chosen my step direction incorrectly?
I'm trying to run a basic gradient descent algorithm with a absolute loss function. I can get it to converge to a good solution by it requires a much lower step size and more iterations than had I ...
5
votes
1answer
1k views
Gradient of Hinge loss
I'm trying to implement basic gradient descent and I'm testing it with a hinge loss function ie $\max(0,1-y\ \mathbf{w.x})$. However, I'm confused about the gradient of the hinge loss. I'm under the ...
7
votes
5answers
759 views
What is the 'fundamental' idea of machine learning for estimating parameters?
The 'fundamental' idea of statistics for estimating parameters is maximum likelihood. I am wondering what is the corresponding idea in machine learning.
Qn 1. Would it be fair to say that the ...
9
votes
4answers
223 views
Comprehensive overview of loss functions?
I am trying to get a global perspective on some of the essential ideas in machine learning, and I was wondering if there is a comprehensive treatment of the different notions of loss (squared, log, ...
